09/14/2026 02:11:36.166 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:11:36.166 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:11:36.166 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:11:36.173 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:11:36.173 7fd3e08452c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:11:36.173 7fd3e08452c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:11:36.174 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:11:36.174 7fd3e08457c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:11:36.174 7fd3e08457c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:11:36.174 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:11:36.174 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.175 7fd3e08457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:11:36.175 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.175 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.175 7fd3e08457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789351896174632 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:11:36.175 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.175 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.176 7fd3e08457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:11:36.176 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.176 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.176 7fd3e08457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:11:36.176 7fd3e08457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.176 7fd3e07f42c0 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:11:36.177 7fd3e07f42c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:11:36.178 7fd3e0845e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:11:36.178 7fd3e07f49a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.178 7fd3e07f49a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:11:36.178 7fd3e07f49a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.178 7fd3e07f49a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.179 7fd3e07f49a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:11:36.179 7fd3e07f49a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.229 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:11:36.236 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:11:36.237 7fd3e0770180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.237 7fd3e0770180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:11:36.237 7fd3e0770180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.237 7fd3e0770180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.237 7fd3e0770180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:11:36.237 7fd3e0770180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.238 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:36.238 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:36.239 7fd3e07f4b80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.239 7fd3e07f4b80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:11:36.239 7fd3e07f4b80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.240 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:11:36.241 7fd3e07f4b80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.241 7fd3e07f4b80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:11:36.241 7fd3e07f4b80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.249 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:11:36.250 7fd3e0770c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.250 7fd3e0770c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:11:36.250 7fd3e0770c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.252 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:11:36.252 7fd3e0770c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.253 7fd3e0770c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:11:36.253 7fd3e0770c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.253 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:11:36.253 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:11:36.253 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:11:36.253 7fd3e0770f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:11:36.254 7fd3e06cb220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.254 7fd3e06cb220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:11:36.254 7fd3e06cb220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.254 7fd3e06cb220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.254 7fd3e06cb220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:11:36.254 7fd3e06cb220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.256 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:11:36.259 7fd3e0770f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:11:36.259 7fd3e06cbae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.259 7fd3e06cbae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:11:36.259 7fd3e06cbae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.280 7fd3e08457c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:11:36.281 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.281 7fd3e06cbe00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:11:36.281 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:36.357 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:36.358 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:36.511 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:36.511 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:36.738 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:36.739 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:37.025 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:37.025 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:37.178 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:37.415 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:37.416 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:37.954 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:37.954 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:38.178 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:39.180 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:39.413 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:39.414 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:40.174 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:11:40.176 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:11:40.176 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:11:40.176 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:11:40.176 7fd3e06cbe00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:11:40.176 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:11:40.176 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:11:40.176 7fd3e06cbe00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:11:40.178 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:11:40.178 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:11:40.178 7fd3e06cbe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:11:40.178 7fd3e06cb720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:11:40.179 7fd3e06cb720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:11:40.182 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:41.182 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:42.183 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:43.185 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:43.292 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:43.293 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:44.186 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:45.187 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:46.188 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:47.189 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:48.191 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:48.485 7fd3e339e220 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:11:48.486 7fd3e339e220 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:11:49.192 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:50.193 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:51.194 7fd3e07f4400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:11:51.198 7fd3e339e220 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:11:51.198 7fd3e339e220 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:11:51.199 7fd3e07f49a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:11:51.345 7fd3e08457c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:11:51.345 7fd3e5615dc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:11:51.346 7fd3e5615dc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:11:51.346 7fd3e339e4a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:11:51.346 7fd3e339e4a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:11:51.353 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:11:51.354 7fd3e5615dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:11:59.930 7f61719d69c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:11:59.930 7f61719d69c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:11:59.930 7f61719d69c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:11:59.933 7f61719d69c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:11:59.934 7f616c98d2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:11:59.934 7f616c98d2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:11:59.934 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:11:59.934 7f616c98d7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:11:59.935 7f616c98d7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:11:59.935 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:11:59.935 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.935 7f616c98d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:11:59.935 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.935 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.936 7f616c98d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789351919935053 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:11:59.936 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.936 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.936 7f616c98d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:11:59.936 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.936 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.936 7f616c98d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:11:59.937 7f616c98d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.937 7f616c98df40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:11:59.937 7f616c98df40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:11:59.938 7f616c98de00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:11:59.939 7f616c93c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.939 7f616c93c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:11:59.939 7f616c93c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.939 7f616c93c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.939 7f616c93c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:11:59.939 7f616c93c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:11:59.994 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:12:00.002 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:12:00.003 7f616c80d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.003 7f616c80d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:00.003 7f616c80d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.003 7f616c80d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.004 7f616c80d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:00.004 7f616c80d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.004 7f616c93cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:00.005 7f616c93c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.005 7f616c93c9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:00.006 7f616c93c9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.006 7f616c93cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:00.007 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:12:00.007 7f616c93c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.007 7f616c93c9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:00.007 7f616c93c9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.019 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:12:00.020 7f616c80dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.021 7f616c80dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:00.021 7f616c80dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.027 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:12:00.028 7f616c80dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.028 7f616c80dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:00.028 7f616c80dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.029 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:12:00.029 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:12:00.030 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:12:00.030 7f616c80df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:12:00.036 7f616c91b220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.037 7f616c91b220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:12:00.037 7f616c91b220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.037 7f616c91b220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.037 7f616c91b220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:12:00.037 7f616c91b220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.042 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:12:00.068 7f616c80df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:12:00.069 7f616c91bd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.070 7f616c91bd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:12:00.070 7f616c91bd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.114 7f616c93cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:00.116 7f616c98d7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:12:00.117 7f616c93cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:00.119 7f616c91bf40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.119 7f616c91bf40 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:12:00.119 7f616c91bf40 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:00.296 7f616c93cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:00.297 7f616c93cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:00.770 7f616c93cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:00.770 7f616c93cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:00.939 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:01.890 7f616c93cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:01.891 7f616c93cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:01.939 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:02.941 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:03.935 7f616c91bf40 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:12:03.936 7f616c91bf40 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:12:03.936 7f616c91bf40 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:12:03.936 7f616c91bf40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:12:03.936 7f616c91bf40 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:12:03.936 7f616c91bf40 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:12:03.936 7f616c91bf40 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:12:03.937 7f616c91bf40 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:12:03.938 7f616c91bf40 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:12:03.938 7f616c91bf40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:12:03.938 7f616c91bf40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:12:03.938 7f616ea88360 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:12:03.939 7f616ea88360 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:12:03.942 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:04.526 7f616c93cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:04.527 7f616c93cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:04.943 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:05.944 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:06.944 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:07.946 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:08.946 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:09.947 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:10.748 7f616c93cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:10.748 7f616c93cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:10.974 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:11.976 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:12.977 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:13.978 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:14.967 7f616c93cc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:12:14.968 7f616c93cc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:12:14.968 7f616c93cc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:12:14.980 7f616c93c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:15.206 7f616c98d7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:12:15.206 7f61719d69c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:12:15.206 7f61719d69c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:12:15.206 7f616c93cb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:12:15.207 7f616c93cb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:12:15.264 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:12:15.264 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:12:15.264 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:12:15.264 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:12:15.265 7f61719d69c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:12:24.072 7f7e4c451700 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:12:24.073 7f7e4c451700 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:12:24.073 7f7e4c451700 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:12:24.077 7f7e4c451700 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:12:24.078 7f7e479c12c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:12:24.078 7f7e479c12c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:12:24.078 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:12:24.078 7f7e479c17c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:12:24.079 7f7e479c17c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:12:24.079 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:12:24.079 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.079 7f7e479c17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:12:24.079 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.079 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.079 7f7e479c17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789351944079012 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:12:24.079 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.080 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.080 7f7e479c17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:12:24.080 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.080 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.080 7f7e479c17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:24.080 7f7e479c17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.081 7f7e479c1e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:12:24.081 7f7e479c1e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:12:24.082 7f7e479c1c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:12:24.082 7f7e47971540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.083 7f7e47971540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:12:24.083 7f7e47971540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.083 7f7e47971540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.083 7f7e47971540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:24.083 7f7e47971540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.162 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:12:24.173 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:12:24.174 7f7e47911040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.175 7f7e47911040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:24.175 7f7e47911040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.175 7f7e47911040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.175 7f7e47911040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:24.175 7f7e47911040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.175 7f7e47971a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:24.176 7f7e47971a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:24.177 7f7e479717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.177 7f7e479717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:24.177 7f7e479717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.184 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:12:24.184 7f7e479717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.185 7f7e479717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:24.185 7f7e479717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.195 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:12:24.196 7f7e47911ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.196 7f7e47911ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:24.196 7f7e47911ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.199 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:12:24.199 7f7e47911ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.199 7f7e47911ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:24.199 7f7e47911ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.199 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:12:24.199 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:12:24.200 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:12:24.200 7f7e47911e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:12:24.200 7f7e478450e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.200 7f7e478450e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:12:24.201 7f7e478450e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.201 7f7e478450e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.201 7f7e478450e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:12:24.201 7f7e478450e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.203 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:12:24.209 7f7e47911e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:12:24.209 7f7e47845c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.209 7f7e47845c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:12:24.209 7f7e47845c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.227 7f7e479c17c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:12:24.228 7f7e47845e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.229 7f7e47845e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:12:24.229 7f7e47845e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:24.315 7f7e47971a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:24.317 7f7e47971a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:24.484 7f7e47971a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:24.485 7f7e47971a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:24.810 7f7e47971a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:24.821 7f7e47971a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:25.082 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:25.589 7f7e47971a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:25.589 7f7e47971a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:26.083 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:27.087 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:27.130 7f7e47971a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:27.131 7f7e47971a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:28.079 7f7e47845e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:12:28.080 7f7e47845e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:12:28.080 7f7e47845e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:12:28.080 7f7e47845e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:12:28.081 7f7e47845e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:12:28.081 7f7e47845e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:12:28.081 7f7e47845e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:12:28.081 7f7e47845e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:12:28.083 7f7e47845e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:12:28.083 7f7e47845e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:12:28.083 7f7e47845e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:12:28.083 7f7e478455e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:12:28.084 7f7e478455e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:12:28.087 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:29.089 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:29.475 7f7e47971a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:29.476 7f7e47971a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:30.090 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:31.092 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:32.093 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:33.095 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:34.096 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:34.589 7f7e47971a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:34.590 7f7e47971a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:35.098 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:36.099 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:37.100 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:38.101 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:39.103 7f7e479c1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:39.108 7f7e47971a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:12:39.109 7f7e47971a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:12:39.109 7f7e47971a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:12:39.319 7f7e479c17c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:12:39.319 7f7e4c451700 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:12:39.320 7f7e4c451700 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:12:39.320 7f7e479719a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:12:39.320 7f7e479719a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:12:39.327 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:12:39.327 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:12:39.327 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:12:39.327 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:12:39.327 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:12:39.327 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:12:39.327 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:12:39.328 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:12:39.328 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:12:39.328 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:12:39.328 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:12:39.328 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:12:39.328 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:12:39.328 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:12:39.328 7f7e4c451700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:12:48.007 7f220affd980 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:12:48.007 7f220affd980 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:12:48.007 7f220affd980 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:12:48.011 7f220affd980 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:12:48.011 7f22065842c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:12:48.012 7f22065842c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:12:48.012 7f22065847c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:12:48.012 7f22065847c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:12:48.013 7f22065847c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:12:48.014 7f22065847c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:12:48.014 7f22065847c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.014 7f22065847c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:12:48.014 7f22065847c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.014 7f22065847c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.014 7f22065847c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789351968013864 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:12:48.014 7f22065847c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.015 7f22065847c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.015 7f22065847c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:12:48.015 7f22065847c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.015 7f22065847c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.015 7f22065847c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:48.015 7f22065847c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.017 7f2206584f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:12:48.017 7f2206584f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:12:48.019 7f2206584e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:12:48.019 7f22065347c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.019 7f22065347c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:12:48.019 7f22065347c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.019 7f22065347c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.019 7f22065347c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:48.020 7f22065347c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.075 7f22065847c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:12:48.083 7f22065847c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:12:48.084 7f220651d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.084 7f220651d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:48.084 7f220651d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.084 7f220651d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.084 7f220651d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:48.084 7f220651d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.085 7f2206534c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:48.085 7f2206534c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:48.086 7f22065349a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.086 7f22065349a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:48.086 7f22065349a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.087 7f22065847c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:12:48.088 7f22065349a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.088 7f22065349a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:48.088 7f22065349a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.097 7f22065847c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:12:48.097 7f220651dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.098 7f220651dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:12:48.098 7f220651dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.100 7f22065847c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:12:48.100 7f220651dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.100 7f220651dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:12:48.101 7f220651dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.101 7f22065847c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:12:48.101 7f22065847c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:12:48.101 7f22065847c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:12:48.101 7f220651df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:12:48.102 7f2207b5e220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.102 7f2207b5e220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:12:48.102 7f2207b5e220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.102 7f2207b5e220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.102 7f2207b5e220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:12:48.102 7f2207b5e220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.105 7f22065847c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:12:48.108 7f220651df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:12:48.109 7f2207b5eae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.109 7f2207b5eae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:12:48.109 7f2207b5eae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.128 7f22065847c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:12:48.130 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.130 7f2207b5ee00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:12:48.130 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:12:48.182 7f2206534c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:48.183 7f2206534c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:48.365 7f2206534c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:48.365 7f2206534c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:48.846 7f2206534c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:48.847 7f2206534c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:49.019 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:49.631 7f2206534c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:49.632 7f2206534c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:50.020 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:50.836 7f2206534c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:50.837 7f2206534c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:51.021 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:52.013 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:12:52.014 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:12:52.014 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:12:52.014 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:12:52.014 7f2207b5ee00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:12:52.014 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:12:52.014 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:12:52.015 7f2207b5ee00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:12:52.016 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:12:52.016 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:12:52.016 7f2207b5ee00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:12:52.016 7f2207b5e720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:12:52.017 7f2207b5e720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:12:52.023 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:53.023 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:54.025 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:54.098 7f2206534c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:54.099 7f2206534c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:55.026 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:56.027 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:57.028 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:58.029 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:12:58.703 7f2206534c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:12:58.704 7f2206534c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:12:59.030 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:00.031 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:01.033 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:02.034 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:03.033 7f2206534c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:13:03.033 7f2206534c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:13:03.034 7f2206534c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:13:03.035 7f22065342c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:03.162 7f22065847c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:13:03.162 7f220affd980 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:13:03.163 7f220affd980 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:13:03.163 7f2206534b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:13:03.163 7f2206534b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:13:03.170 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:13:03.171 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:13:03.171 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:13:03.171 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:13:03.171 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:13:03.171 7f220affd980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:13:11.647 7fc1c2da8480 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:13:11.647 7fc1c2da8480 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:13:11.647 7fc1c2da8480 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:13:11.650 7fc1c2da8480 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:13:11.651 7fc1bdfe52c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:13:11.651 7fc1bdfe52c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:13:11.651 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:13:11.651 7fc1bdfe57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:13:11.652 7fc1bdfe57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:13:11.652 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:13:11.652 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.652 7fc1bdfe57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:13:11.652 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.653 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.653 7fc1bdfe57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789351991652195 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:13:11.653 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.653 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.653 7fc1bdfe57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:13:11.654 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.654 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.654 7fc1bdfe57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:11.654 7fc1bdfe57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.654 7fc1bdfe5e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:13:11.654 7fc1bdfe5e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:13:11.655 7fc1bdfe5c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:13:11.656 7fc1bdf94540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.656 7fc1bdf94540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:13:11.656 7fc1bdf94540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.656 7fc1bdf94540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.656 7fc1bdf94540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:11.656 7fc1bdf94540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.710 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:13:11.717 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:13:11.718 7fc1c00470e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.718 7fc1c00470e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:11.718 7fc1c00470e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.718 7fc1c00470e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.718 7fc1c00470e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:11.718 7fc1c00470e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.719 7fc1bdf94c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:11.719 7fc1bdf94c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:11.720 7fc1bdf947c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.721 7fc1bdf947c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:11.721 7fc1bdf947c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.722 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:13:11.722 7fc1bdf947c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.722 7fc1bdf947c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:11.722 7fc1bdf947c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.732 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:13:11.732 7fc1c0047c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.733 7fc1c0047c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:11.733 7fc1c0047c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.735 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:13:11.736 7fc1c0047c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.736 7fc1c0047c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:11.736 7fc1c0047c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.736 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:13:11.736 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:13:11.736 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:13:11.737 7fc1c0047f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:13:11.737 7fc1bde68220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.737 7fc1bde68220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:13:11.737 7fc1bde68220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.737 7fc1bde68220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.737 7fc1bde68220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:13:11.738 7fc1bde68220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.740 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:13:11.743 7fc1c0047f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:13:11.744 7fc1bde68ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.744 7fc1bde68ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:13:11.744 7fc1bde68ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.762 7fc1bdfe57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:13:11.763 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.763 7fc1bde68e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:13:11.764 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:11.843 7fc1bdf94c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:11.844 7fc1bdf94c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:12.050 7fc1bdf94c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:12.074 7fc1bdf94c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:12.355 7fc1bdf94c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:12.355 7fc1bdf94c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:12.656 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:13.125 7fc1bdf94c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:13.126 7fc1bdf94c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:13.657 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:14.319 7fc1bdf94c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:14.320 7fc1bdf94c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:14.659 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:15.652 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:13:15.653 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:13:15.654 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:13:15.654 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:13:15.654 7fc1bde68e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:13:15.654 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:13:15.654 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:13:15.654 7fc1bde68e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:13:15.656 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:13:15.656 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:13:15.656 7fc1bde68e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:13:15.656 7fc1bde68720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:13:15.657 7fc1bde68720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:13:15.660 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:16.490 7fc1bdf94c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:16.491 7fc1bdf94c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:16.661 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:17.662 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:18.663 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:19.664 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:20.419 7fc1bdf94c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:20.420 7fc1bdf94c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:20.665 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:21.666 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:22.667 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:23.668 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:24.669 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:25.670 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:26.671 7fc1bdfe5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:26.749 7fc1bdf94c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:13:26.749 7fc1bdf94c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:13:26.750 7fc1be787c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:13:26.769 7fc1bdfe57c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:13:26.769 7fc1c2da8480 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:13:26.769 7fc1c2da8480 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:13:26.770 7fc1bdf94680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:13:26.770 7fc1bdf94680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:13:26.889 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:13:26.890 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:13:26.890 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:13:26.890 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:13:26.890 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:13:26.890 7fc1c2da8480 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:13:35.485 7fb1feb82180 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:13:35.485 7fb1feb82180 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:13:35.485 7fb1feb82180 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:13:35.489 7fb1feb82180 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:13:35.490 7fb1fa0f62c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:13:35.490 7fb1fa0f62c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:13:35.490 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:13:35.491 7fb1fa0f67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:13:35.491 7fb1fa0f67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:13:35.491 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:13:35.491 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.492 7fb1fa0f67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:13:35.492 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.492 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.492 7fb1fa0f67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352015491472 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:13:35.492 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.492 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.493 7fb1fa0f67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:13:35.493 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.493 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.493 7fb1fa0f67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:35.493 7fb1fa0f67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.493 7fb1fa0f6e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:13:35.494 7fb1fa0f6e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:13:35.495 7fb1fa0f6c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:13:35.495 7fb1fa0a6540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.495 7fb1fa0a6540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:13:35.495 7fb1fa0a6540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.495 7fb1fa0a6540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.496 7fb1fa0a6540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:35.496 7fb1fa0a6540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.553 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:13:35.562 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:13:35.563 7fb1fa046040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.563 7fb1fa046040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:35.563 7fb1fa046040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.563 7fb1fa046040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.563 7fb1fa046040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:35.563 7fb1fa046040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.564 7fb1fa0a6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:35.565 7fb1fa0a6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:35.565 7fb1fa0a67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.566 7fb1fa0a67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:35.566 7fb1fa0a67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.567 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:13:35.567 7fb1fa0a67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.567 7fb1fa0a67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:35.567 7fb1fa0a67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.577 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:13:35.577 7fb1fa046ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.577 7fb1fa046ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:35.578 7fb1fa046ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.580 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:13:35.580 7fb1fa046ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.580 7fb1fa046ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:35.580 7fb1fa046ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.581 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:13:35.581 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:13:35.581 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:13:35.581 7fb1fa046e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:13:35.582 7fb1f9f7a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.582 7fb1f9f7a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:13:35.582 7fb1f9f7a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.582 7fb1f9f7a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.582 7fb1f9f7a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:13:35.582 7fb1f9f7a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.585 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:13:35.588 7fb1fa046e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:13:35.589 7fb1f9f7a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.589 7fb1f9f7a9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:13:35.589 7fb1f9f7a9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.613 7fb1fa0f67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:13:35.617 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.618 7fb1f9f7acc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:13:35.618 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:35.676 7fb1fa0a6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:35.677 7fb1fa0a6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:35.886 7fb1fa0a6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:35.887 7fb1fa0a6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:36.343 7fb1fa0a6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:36.344 7fb1fa0a6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:36.495 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:37.433 7fb1fa0a6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:37.434 7fb1fa0a6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:37.496 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:38.497 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:38.839 7fb1fa0a6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:38.839 7fb1fa0a6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:39.491 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:13:39.493 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:13:39.493 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:13:39.493 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:13:39.493 7fb1f9f7acc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:13:39.493 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:13:39.493 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:13:39.493 7fb1f9f7acc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:13:39.495 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:13:39.495 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:13:39.495 7fb1f9f7acc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:13:39.495 7fb1f9f7a5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:13:39.496 7fb1f9f7a5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:13:39.498 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:40.499 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:41.500 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:42.501 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:42.546 7fb1fa0a6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:42.547 7fb1fa0a6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:43.502 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:44.503 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:45.504 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:46.505 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:47.506 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:48.507 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:49.508 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:49.628 7fb1fa0a6a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:49.629 7fb1fa0a6a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:50.510 7fb1fa0f6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:13:50.533 7fb1fa0a6a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:13:50.534 7fb1fa0a6a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:13:50.534 7fb1fa0a6a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:13:50.706 7fb1fa0f67c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:13:50.707 7fb1feb82180 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:13:50.707 7fb1feb82180 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:13:50.707 7fb1fa0a69a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:13:50.707 7fb1fa0a69a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:13:50.715 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:13:50.715 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:13:50.715 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:13:50.715 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:13:50.715 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:13:50.715 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:13:50.715 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:13:50.716 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:13:50.716 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:13:50.716 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:13:50.716 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:13:50.716 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:13:50.716 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:13:50.716 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:13:50.716 7fb1feb82180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:13:59.665 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:13:59.665 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:13:59.666 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:13:59.669 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:13:59.669 7fa1aec9e2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:13:59.669 7fa1aec9e2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:13:59.669 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:13:59.670 7fa1aec9e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:13:59.670 7fa1aec9e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:13:59.670 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:13:59.670 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.670 7fa1aec9e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:13:59.670 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.670 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.670 7fa1aec9e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352039670368 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:13:59.670 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.671 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.671 7fa1aec9e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:13:59.671 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.671 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.671 7fa1aec9e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:59.671 7fa1aec9e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.672 7fa1aec9ee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:13:59.672 7fa1aec9ee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:13:59.673 7fa1aec9ec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:13:59.673 7fa1aec4d540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.673 7fa1aec4d540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:13:59.673 7fa1aec4d540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.673 7fa1aec4d540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.673 7fa1aec4d540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:59.674 7fa1aec4d540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.726 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:13:59.734 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:13:59.735 7fa1b1288040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.735 7fa1b1288040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:59.735 7fa1b1288040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.735 7fa1b1288040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.735 7fa1b1288040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:59.735 7fa1b1288040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.735 7fa1aec4da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:59.736 7fa1aec4da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:13:59.737 7fa1aec4d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.737 7fa1aec4d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:59.737 7fa1aec4d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.739 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:13:59.739 7fa1aec4d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.739 7fa1aec4d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:59.739 7fa1aec4d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.749 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:13:59.750 7fa1b1288ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.750 7fa1b1288ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:13:59.750 7fa1b1288ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.752 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:13:59.753 7fa1b1288ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.753 7fa1b1288ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:13:59.753 7fa1b1288ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.753 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:13:59.753 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:13:59.754 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:13:59.754 7fa1b1288e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:13:59.754 7fa1aeb210e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.754 7fa1aeb210e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:13:59.755 7fa1aeb210e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.755 7fa1aeb210e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.755 7fa1aeb210e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:13:59.755 7fa1aeb210e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.757 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:13:59.760 7fa1b1288e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:13:59.760 7fa1aeb219a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.760 7fa1aeb219a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:13:59.761 7fa1aeb219a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.777 7fa1aec9e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:13:59.778 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.778 7fa1aeb21cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:13:59.778 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:13:59.874 7fa1aec4da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:13:59.874 7fa1aec4da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:00.065 7fa1aec4da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:00.066 7fa1aec4da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:00.503 7fa1aec4da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:00.504 7fa1aec4da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:00.673 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:01.637 7fa1aec4da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:01.638 7fa1aec4da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:01.673 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:02.674 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:03.670 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:14:03.671 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:14:03.672 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:14:03.672 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:14:03.672 7fa1aeb21cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:14:03.672 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:14:03.672 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:14:03.672 7fa1aeb21cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:14:03.674 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:14:03.674 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:14:03.674 7fa1aeb21cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:14:03.674 7fa1aeb215e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:14:03.675 7fa1aeb215e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:14:03.676 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:04.122 7fa1aec4da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:04.123 7fa1aec4da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:04.677 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:05.678 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:06.679 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:07.681 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:08.683 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:09.479 7fa1aec4da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:09.480 7fa1aec4da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:09.684 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:10.685 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:11.687 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:12.688 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:13.689 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:14.691 7fa1aec9ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:14.703 7fa1aec4da40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:14:14.704 7fa1aec4da40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:14:14.704 7fa1aec4da40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:14:14.828 7fa1aec9e7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:14:14.828 7fa1b3a6cd80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:14:14.828 7fa1b3a6cd80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:14:14.828 7fa1aec4d9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:14:14.828 7fa1aec4d9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:14:14.836 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:14:14.837 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:14:14.837 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:14:14.837 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:14:14.837 7fa1b3a6cd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:14:23.275 7f6640efa140 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:14:23.275 7f6640efa140 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:14:23.275 7f6640efa140 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:14:23.278 7f6640efa140 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:14:23.279 7f663c47a2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:14:23.279 7f663c47a2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:14:23.279 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:14:23.280 7f663c47a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:14:23.280 7f663c47a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:14:23.280 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:14:23.280 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.280 7f663c47a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:14:23.280 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.281 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.281 7f663c47a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352063280441 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:14:23.281 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.281 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.281 7f663c47a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:14:23.281 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.281 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.282 7f663c47a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:14:23.282 7f663c47a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.282 7f663c47ae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:14:23.282 7f663c47ae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:14:23.283 7f663c47ac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:14:23.284 7f663c42a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.284 7f663c42a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:14:23.284 7f663c42a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.284 7f663c42a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.284 7f663c42a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:14:23.284 7f663c42a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.338 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:14:23.345 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:14:23.346 7f663c3cd040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.346 7f663c3cd040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:14:23.346 7f663c3cd040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.346 7f663c3cd040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.347 7f663c3cd040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:14:23.347 7f663c3cd040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.347 7f663c42aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:23.348 7f663c42aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:23.349 7f663c42a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.349 7f663c42a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:14:23.349 7f663c42a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.350 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:14:23.350 7f663c42a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.350 7f663c42a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:14:23.350 7f663c42a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.360 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:14:23.360 7f663c3cdae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.360 7f663c3cdae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:14:23.361 7f663c3cdae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.363 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:14:23.363 7f663c3cdae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.363 7f663c3cdae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:14:23.363 7f663c3cdae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.363 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:14:23.364 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:14:23.364 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:14:23.364 7f663c3cde00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:14:23.364 7f663c3000e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.365 7f663c3000e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:14:23.365 7f663c3000e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.365 7f663c3000e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.365 7f663c3000e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:14:23.365 7f663c3000e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.367 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:14:23.370 7f663c3cde00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:14:23.371 7f663c3009a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.371 7f663c3009a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:14:23.371 7f663c3009a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.387 7f663c47a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:14:23.389 7f663c300cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.390 7f663c300cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:14:23.390 7f663c300cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:23.448 7f663c42aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:23.449 7f663c42aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:23.457 7f663c300cc0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:14:23.577 7f663c42aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:23.578 7f663c42aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:23.732 7f663c42aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:23.733 7f663c42aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:24.058 7f663c42aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:24.058 7f663c42aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:24.283 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:24.858 7f663c42aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:24.859 7f663c42aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:25.284 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:26.285 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:26.422 7f663c42aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:26.423 7f663c42aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:27.280 7f663c3cdd60 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:14:27.281 7f663c3cdd60 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:14:27.282 7f663c3cdd60 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:14:27.282 7f663c3cdd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:14:27.282 7f663c3cdd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:14:27.282 7f663c3cdd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:14:27.282 7f663c3cdd60 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:14:27.282 7f663c3cdd60 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:14:27.282 7f663c3cdd60 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:14:27.283 7f663c3cdd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:14:27.283 7f663c3cdd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:14:27.283 7f663c3007c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:14:27.283 7f663c3007c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:14:27.286 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:28.287 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:29.288 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:29.722 7f663c42aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:29.746 7f663c42aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:30.289 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:31.290 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:32.291 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:33.293 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:34.294 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:35.295 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:36.296 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:37.297 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:37.382 7f663c42aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:37.383 7f663c42aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:38.287 7f663c42aa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:14:38.287 7f663c42aa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:14:38.288 7f663c42aa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:14:38.298 7f663c47af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:38.460 7f663c47a7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:14:38.461 7f6640efa140 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:14:38.461 7f6640efa140 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:14:38.461 7f663c42a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:14:38.461 7f663c42a9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:14:38.469 7f6640efa140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:14:46.933 7fe71724e940 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:14:46.933 7fe71724e940 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:14:46.933 7fe71724e940 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:14:46.937 7fe71724e940 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:14:46.937 7fe7127d92c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:14:46.937 7fe7127d92c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:14:46.938 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:14:46.938 7fe7127d97c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:14:46.938 7fe7127d97c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:14:46.939 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:14:46.939 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.939 7fe7127d97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:14:46.939 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.939 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.939 7fe7127d97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352086938811 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:14:46.939 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.939 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.940 7fe7127d97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:14:46.940 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.940 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.940 7fe7127d97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:14:46.940 7fe7127d97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.940 7fe7127d9e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:14:46.941 7fe7127d9e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:14:46.942 7fe7127d9c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:14:46.942 7fe712788540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.942 7fe712788540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:14:46.942 7fe712788540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.942 7fe712788540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.943 7fe712788540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:14:46.943 7fe712788540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:46.996 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:14:47.003 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:14:47.004 7fe7120cd0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.005 7fe7120cd0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:14:47.005 7fe7120cd0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.005 7fe7120cd0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.005 7fe7120cd0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:14:47.005 7fe7120cd0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.005 7fe712788c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:47.006 7fe712788c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:47.007 7fe7127887c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.007 7fe7127887c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:14:47.008 7fe7127887c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.009 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:14:47.009 7fe7127887c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.009 7fe7127887c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:14:47.009 7fe7127887c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.018 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:14:47.019 7fe7120cdc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.019 7fe7120cdc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:14:47.019 7fe7120cdc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.022 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:14:47.022 7fe7120cdc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.022 7fe7120cdc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:14:47.022 7fe7120cdc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.022 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:14:47.022 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:14:47.023 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:14:47.023 7fe7120cdf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:14:47.023 7fe71274c220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.024 7fe71274c220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:14:47.024 7fe71274c220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.024 7fe71274c220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.024 7fe71274c220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:14:47.024 7fe71274c220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.026 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:14:47.029 7fe7120cdf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:14:47.030 7fe71274cae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.030 7fe71274cae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:14:47.030 7fe71274cae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.049 7fe7127d97c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:14:47.050 7fe71274ce00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.051 7fe71274ce00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:14:47.051 7fe71274ce00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:14:47.114 7fe712788c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:47.115 7fe712788c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:47.367 7fe712788c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:47.367 7fe712788c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:47.814 7fe712788c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:47.815 7fe712788c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:47.942 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:48.942 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:48.955 7fe712788c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:48.955 7fe712788c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:49.944 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:50.938 7fe71274ce00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:14:50.940 7fe71274ce00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:14:50.940 7fe71274ce00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:14:50.940 7fe71274ce00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:14:50.940 7fe71274ce00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:14:50.940 7fe71274ce00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:14:50.940 7fe71274ce00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:14:50.941 7fe71274ce00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:14:50.942 7fe71274ce00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:14:50.942 7fe71274ce00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:14:50.943 7fe71274ce00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:14:50.943 7fe71274c720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:14:50.944 7fe71274c720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:14:50.944 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:51.280 7fe71274c720 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:14:51.669 7fe712788c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:51.672 7fe712788c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:51.946 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:52.946 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:53.947 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:54.948 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:55.787 7fe712788c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:14:55.788 7fe712788c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:14:55.949 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:56.950 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:57.951 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:58.952 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:14:59.954 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:00.955 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:01.956 7fe7127d9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:02.018 7fe712788c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:15:02.018 7fe712788c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:15:02.018 7fe712f7bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:15:02.151 7fe7127d97c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:15:02.151 7fe71724e940 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:15:02.152 7fe71724e940 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:15:02.152 7fe712788680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:15:02.152 7fe712788680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:15:02.159 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:15:02.159 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:15:02.160 7fe71724e940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:15:10.690 7f9ab6677a00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:15:10.690 7f9ab6677a00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:15:10.691 7f9ab6677a00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:15:10.694 7f9ab6677a00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:15:10.694 7f9ab16152c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:15:10.695 7f9ab16152c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:15:10.695 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:15:10.695 7f9ab16157c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:15:10.695 7f9ab16157c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:15:10.696 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:15:10.696 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.696 7f9ab16157c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:15:10.696 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.696 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.696 7f9ab16157c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352110695865 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:15:10.696 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.697 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.697 7f9ab16157c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:15:10.697 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.697 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.697 7f9ab16157c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:10.697 7f9ab16157c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.698 7f9ab1615e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:15:10.698 7f9ab1615e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:15:10.699 7f9ab1615c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:15:10.700 7f9ab15c5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.700 7f9ab15c5540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:15:10.700 7f9ab15c5540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.700 7f9ab15c5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.700 7f9ab15c5540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:10.700 7f9ab15c5540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.754 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:15:10.762 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:15:10.763 7f9ab15640e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.763 7f9ab15640e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:10.763 7f9ab15640e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.763 7f9ab15640e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.763 7f9ab15640e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:10.763 7f9ab15640e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.764 7f9ab15c5c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:10.765 7f9ab15c5c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:10.765 7f9ab15c57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.766 7f9ab15c57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:10.766 7f9ab15c57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.767 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:15:10.767 7f9ab15c57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.767 7f9ab15c57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:10.767 7f9ab15c57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.777 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:15:10.778 7f9ab1564c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.778 7f9ab1564c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:10.778 7f9ab1564c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.780 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:15:10.780 7f9ab1564c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.781 7f9ab1564c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:10.781 7f9ab1564c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.781 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:15:10.781 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:15:10.781 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:15:10.782 7f9ab1564f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:15:10.782 7f9ab1498220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.782 7f9ab1498220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:15:10.782 7f9ab1498220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.782 7f9ab1498220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.782 7f9ab1498220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:15:10.782 7f9ab1498220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.785 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:15:10.789 7f9ab1564f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:15:10.789 7f9ab1498ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.789 7f9ab1498ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:15:10.789 7f9ab1498ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.806 7f9ab16157c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:15:10.807 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.808 7f9ab1498e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:15:10.808 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:10.826 7f9ab15c5c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:10.827 7f9ab15c5c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:10.941 7f9ab15c5c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:10.942 7f9ab15c5c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:11.105 7f9ab15c5c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:11.106 7f9ab15c5c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:11.528 7f9ab15c5c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:11.528 7f9ab15c5c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:11.699 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:12.683 7f9ab15c5c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:12.683 7f9ab15c5c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:12.701 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:13.702 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:14.695 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:15:14.697 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:15:14.697 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:15:14.697 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:15:14.697 7f9ab1498e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:15:14.697 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:15:14.697 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:15:14.697 7f9ab1498e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:15:14.699 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:15:14.699 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:15:14.699 7f9ab1498e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:15:14.699 7f9ab1498720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:15:14.700 7f9ab1498720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:15:14.703 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:15.042 7f9ab15c5c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:15.042 7f9ab15c5c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:15.704 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:16.705 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:17.706 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:18.706 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:18.720 7f9ab1db7c20 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:15:19.708 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:20.708 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:20.917 7f9ab15c5c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:20.918 7f9ab15c5c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:21.710 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:22.712 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:23.712 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:24.714 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:25.715 7f9ab1615f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:25.739 7f9ab15c5c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:15:25.739 7f9ab15c5c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:15:25.740 7f9ab15c5c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:15:25.856 7f9ab16157c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:15:25.857 7f9ab6677a00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:15:25.857 7f9ab6677a00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:15:25.857 7f9ab15c5b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:15:25.857 7f9ab15c5b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:15:25.864 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:15:25.865 7f9ab6677a00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:15:34.344 7f35f1045680 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:15:34.344 7f35f1045680 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:15:34.344 7f35f1045680 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:15:34.347 7f35f1045680 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:15:34.347 7f35ec5b62c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:15:34.347 7f35ec5b62c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:15:34.348 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:15:34.348 7f35ec5b67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:15:34.348 7f35ec5b67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:15:34.349 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:15:34.349 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.349 7f35ec5b67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:15:34.349 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.349 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.349 7f35ec5b67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352134348764 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:15:34.349 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.349 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.350 7f35ec5b67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:15:34.350 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.350 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.350 7f35ec5b67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:34.350 7f35ec5b67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.350 7f35ec5b6e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:15:34.350 7f35ec5b6e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:15:34.351 7f35ec5b6c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:15:34.352 7f35ec565540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.352 7f35ec565540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:15:34.352 7f35ec565540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.352 7f35ec565540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.352 7f35ec565540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:34.352 7f35ec565540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.406 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:15:34.414 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:15:34.415 7f35ec50a040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.415 7f35ec50a040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:34.415 7f35ec50a040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.415 7f35ec50a040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.415 7f35ec50a040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:34.415 7f35ec50a040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.416 7f35ec565a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:34.417 7f35ec565a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:34.417 7f35ec5657c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.417 7f35ec5657c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:34.418 7f35ec5657c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.419 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:15:34.419 7f35ec5657c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.419 7f35ec5657c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:34.419 7f35ec5657c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.429 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:15:34.429 7f35ec50aae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.429 7f35ec50aae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:34.429 7f35ec50aae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.432 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:15:34.432 7f35ec50aae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.432 7f35ec50aae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:34.432 7f35ec50aae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.432 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:15:34.433 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:15:34.433 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:15:34.433 7f35ec50ae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:15:34.433 7f35ec43a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.434 7f35ec43a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:15:34.434 7f35ec43a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.434 7f35ec43a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.434 7f35ec43a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:15:34.434 7f35ec43a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.436 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:15:34.439 7f35ec50ae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:15:34.439 7f35ec43a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.439 7f35ec43a9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:15:34.439 7f35ec43a9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.457 7f35ec5b67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:15:34.458 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.459 7f35ec43acc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:15:34.459 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:34.494 7f35ec565a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:34.495 7f35ec565a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:34.629 7f35ec565a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:34.630 7f35ec565a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:34.830 7f35ec565a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:34.831 7f35ec565a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:35.292 7f35ec565a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:35.293 7f35ec565a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:35.352 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:36.232 7f35ec565a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:36.232 7f35ec565a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:36.353 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:37.354 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:37.650 7f35ec565a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:37.651 7f35ec565a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:38.349 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:15:38.350 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:15:38.350 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:15:38.350 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:15:38.350 7f35ec43acc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:15:38.351 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:15:38.351 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:15:38.351 7f35ec43acc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:15:38.353 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:15:38.353 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:15:38.353 7f35ec43acc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:15:38.353 7f35ec43a5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:15:38.354 7f35ec43a5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:15:38.355 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:39.356 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:40.358 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:40.924 7f35ec565a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:40.925 7f35ec565a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:41.359 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:42.360 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:43.362 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:44.363 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:45.364 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:45.963 7f35ec43a5e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:15:46.365 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:47.366 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:48.367 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:48.473 7f35ec565a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:48.474 7f35ec565a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:49.369 7f35ec5b6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:49.377 7f35ec565a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:15:49.377 7f35ec565a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:15:49.378 7f35ec565a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:15:49.477 7f35ec5b67c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:15:49.477 7f35f1045680 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:15:49.478 7f35f1045680 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:15:49.478 7f35ec5659a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:15:49.478 7f35ec5659a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:15:49.485 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:15:49.486 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:15:49.486 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:15:49.486 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:15:49.486 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:15:49.486 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:15:49.486 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:15:49.486 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:15:49.486 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:15:49.486 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:15:49.487 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:15:49.487 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:15:49.487 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:15:49.487 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:15:49.487 7f35f1045680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:15:57.829 7fd68fcaad80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:15:57.830 7fd68fcaad80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:15:57.830 7fd68fcaad80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:15:57.833 7fd68fcaad80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:15:57.833 7fd68aee02c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:15:57.833 7fd68aee02c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:15:57.833 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:15:57.834 7fd68aee07c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:15:57.834 7fd68aee07c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:15:57.834 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:15:57.834 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.834 7fd68aee07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:15:57.835 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.835 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.835 7fd68aee07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352157834541 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:15:57.835 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.835 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.836 7fd68aee07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:15:57.836 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.836 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.836 7fd68aee07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:57.836 7fd68aee07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.837 7fd68aee0f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:15:57.837 7fd68aee0f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:15:57.838 7fd68aee0c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:15:57.838 7fd68ae8f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.838 7fd68ae8f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:15:57.838 7fd68ae8f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.838 7fd68ae8f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.839 7fd68ae8f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:57.839 7fd68ae8f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.894 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:15:57.902 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:15:57.902 7fd68ad580e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.903 7fd68ad580e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:57.903 7fd68ad580e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.903 7fd68ad580e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.903 7fd68ad580e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:57.903 7fd68ad580e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.903 7fd68ae8fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:57.904 7fd68ae8fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:57.905 7fd68ae8f9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.905 7fd68ae8f9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:57.905 7fd68ae8f9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.907 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:15:57.907 7fd68ae8f9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.907 7fd68ae8f9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:57.907 7fd68ae8f9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.917 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:15:57.917 7fd68ad58c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.918 7fd68ad58c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:15:57.918 7fd68ad58c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.920 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:15:57.921 7fd68ad58c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.921 7fd68ad58c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:15:57.921 7fd68ad58c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.921 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:15:57.921 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:15:57.921 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:15:57.922 7fd68ad58f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:15:57.922 7fd68c4ba220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.922 7fd68c4ba220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:15:57.922 7fd68c4ba220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.922 7fd68c4ba220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.922 7fd68c4ba220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:15:57.922 7fd68c4ba220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.925 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:15:57.928 7fd68ad58f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:15:57.929 7fd68c4baae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.929 7fd68c4baae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:15:57.929 7fd68c4baae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.945 7fd68aee07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:15:57.947 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:57.947 7fd68c4bae00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:15:57.947 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:15:58.045 7fd68ae8fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:58.046 7fd68ae8fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:58.237 7fd68ae8fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:58.238 7fd68ae8fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:58.647 7fd68ae8fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:58.648 7fd68ae8fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:58.838 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:15:59.597 7fd68ae8fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:15:59.598 7fd68ae8fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:15:59.840 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:00.841 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:01.834 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:16:01.835 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:16:01.836 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:16:01.836 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:16:01.836 7fd68c4bae00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:16:01.836 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:16:01.836 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:16:01.836 7fd68c4bae00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:16:01.838 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:16:01.838 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:16:01.838 7fd68c4bae00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:16:01.838 7fd68c4ba720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:16:01.839 7fd68c4ba720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:16:01.842 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:02.017 7fd68ae8fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:02.017 7fd68ae8fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:02.843 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:03.844 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:04.846 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:05.846 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:06.847 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:07.849 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:07.918 7fd68ae8fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:07.919 7fd68ae8fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:08.849 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:09.851 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:10.852 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:11.853 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:12.855 7fd68ae8f2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:12.939 7fd68ae8fc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:16:12.939 7fd68ae8fc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:16:12.940 7fd68ae8fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:16:13.001 7fd68aee07c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:16:13.001 7fd68fcaad80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:16:13.001 7fd68fcaad80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:16:13.001 7fd68ae8fb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:16:13.002 7fd68ae8fb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:16:13.009 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:16:13.010 7fd68fcaad80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:16:21.324 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:16:21.324 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:16:21.324 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:16:21.327 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:16:21.328 7fd0ddaf12c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:16:21.328 7fd0ddaf12c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:16:21.328 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:16:21.328 7fd0ddaf17c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:16:21.329 7fd0ddaf17c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:16:21.329 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:16:21.329 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.329 7fd0ddaf17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:16:21.329 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.329 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.329 7fd0ddaf17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352181329024 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:16:21.329 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.330 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.330 7fd0ddaf17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:16:21.330 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.330 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.330 7fd0ddaf17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:16:21.330 7fd0ddaf17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.330 7fd0ddaf1e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:16:21.331 7fd0ddaf1e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:16:21.332 7fd0ddaf1c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:16:21.332 7fd0ddaa1540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.332 7fd0ddaa1540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:16:21.332 7fd0ddaa1540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.332 7fd0ddaa1540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.332 7fd0ddaa1540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:16:21.333 7fd0ddaa1540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.385 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:16:21.392 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:16:21.393 7fd0dda440e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.394 7fd0dda440e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:16:21.394 7fd0dda440e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.394 7fd0dda440e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.394 7fd0dda440e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:16:21.394 7fd0dda440e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.394 7fd0ddaa1c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:21.395 7fd0ddaa1c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:21.396 7fd0ddaa17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.396 7fd0ddaa17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:16:21.397 7fd0ddaa17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.397 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:16:21.398 7fd0ddaa17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.398 7fd0ddaa17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:16:21.398 7fd0ddaa17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.407 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:16:21.408 7fd0dda44c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.408 7fd0dda44c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:16:21.408 7fd0dda44c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.411 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:16:21.411 7fd0dda44c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.411 7fd0dda44c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:16:21.411 7fd0dda44c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.412 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:16:21.412 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:16:21.412 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:16:21.412 7fd0dda44f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:16:21.413 7fd0dd977220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.413 7fd0dd977220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:16:21.413 7fd0dd977220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.413 7fd0dd977220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.413 7fd0dd977220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:16:21.413 7fd0dd977220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.416 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:16:21.421 7fd0dda44f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:16:21.421 7fd0dd977d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.421 7fd0dd977d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:16:21.421 7fd0dd977d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.437 7fd0ddaf17c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:16:21.438 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.439 7fd0dd977f40 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:16:21.439 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:21.528 7fd0ddaa1c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:21.529 7fd0ddaa1c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:21.753 7fd0ddaa1c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:21.754 7fd0ddaa1c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:22.227 7fd0ddaa1c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:22.228 7fd0ddaa1c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:22.331 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:23.332 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:23.450 7fd0ddaa1c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:23.451 7fd0ddaa1c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:24.333 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:25.329 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:16:25.331 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:16:25.331 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:16:25.331 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:16:25.331 7fd0dd977f40 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:16:25.331 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:16:25.331 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:16:25.331 7fd0dd977f40 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:16:25.333 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:16:25.333 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:16:25.333 7fd0dd977f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:16:25.333 7fd0dd977720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:16:25.334 7fd0dd977720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:16:25.335 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:26.336 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:26.815 7fd0ddaa1c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:26.816 7fd0ddaa1c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:27.337 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:28.338 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:29.339 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:30.340 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:31.341 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:32.386 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:33.388 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:33.939 7fd0ddaa1c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:33.940 7fd0ddaa1c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:34.389 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:35.390 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:36.351 7fd0ddaa1c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:16:36.351 7fd0ddaa1c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:16:36.352 7fd0ddaa1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:16:36.392 7fd0ddaf1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:36.453 7fd0ddaf17c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:16:36.454 7fd0e25b26c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:16:36.454 7fd0e25b26c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:16:36.454 7fd0ddaa1b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:16:36.454 7fd0ddaa1b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:16:36.460 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:16:36.461 7fd0e25b26c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:16:44.901 7f22d02c3c80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:16:44.901 7f22d02c3c80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:16:44.901 7f22d02c3c80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:16:44.904 7f22d02c3c80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:16:44.905 7f22cb66f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:16:44.905 7f22cb66f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:16:44.905 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:16:44.906 7f22cb66f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:16:44.906 7f22cb66f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:16:44.906 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:16:44.906 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.906 7f22cb66f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:16:44.906 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.907 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.907 7f22cb66f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352204906515 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:16:44.907 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.907 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.908 7f22cb66f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:16:44.908 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.908 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.908 7f22cb66f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:16:44.908 7f22cb66f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.908 7f22cb66fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:16:44.908 7f22cb66fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:16:44.909 7f22cb66fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:16:44.910 7f22cb61f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.910 7f22cb61f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:16:44.910 7f22cb61f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.910 7f22cb61f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.911 7f22cb61f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:16:44.911 7f22cb61f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.965 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:16:44.973 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:16:44.974 7f22cb5bf0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.974 7f22cb5bf0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:16:44.974 7f22cb5bf0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.974 7f22cb5bf0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.974 7f22cb5bf0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:16:44.975 7f22cb5bf0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.975 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:44.976 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:44.976 7f22cb61f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.977 7f22cb61f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:16:44.977 7f22cb61f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.978 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:16:44.978 7f22cb61f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.978 7f22cb61f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:16:44.978 7f22cb61f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.987 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:16:44.988 7f22cb5bfc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.988 7f22cb5bfc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:16:44.988 7f22cb5bfc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.990 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:16:44.990 7f22cb5bfc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.991 7f22cb5bfc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:16:44.991 7f22cb5bfc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.991 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:16:44.991 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:16:44.991 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:16:44.991 7f22cb5bff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:16:44.992 7f22cb4f3220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.992 7f22cb4f3220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:16:44.992 7f22cb4f3220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.992 7f22cb4f3220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.992 7f22cb4f3220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:16:44.992 7f22cb4f3220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.994 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:16:44.997 7f22cb5bff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:16:44.997 7f22cb4f3ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:44.998 7f22cb4f3ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:16:44.998 7f22cb4f3ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:45.017 7f22cb66f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:16:45.018 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:45.018 7f22cb4f3e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:16:45.018 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:16:45.075 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:45.075 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:45.260 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:45.260 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:45.588 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:45.588 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:45.909 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:46.014 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:46.015 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:46.564 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:46.565 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:46.910 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:47.759 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:47.760 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:47.911 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:48.907 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:16:48.908 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:16:48.908 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:16:48.908 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:16:48.908 7f22cb4f3e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:16:48.909 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:16:48.909 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:16:48.909 7f22cb4f3e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:16:48.911 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:16:48.911 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:16:48.911 7f22cb4f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:16:48.911 7f22cb4f3720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:16:48.912 7f22cb4f3720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:16:48.913 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:49.411 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:49.412 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:49.913 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:50.914 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:51.915 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:52.160 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:52.161 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:52.916 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:53.917 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:54.918 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:55.919 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:56.544 7f22cb61fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:16:56.545 7f22cb61fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:16:56.920 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:57.921 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:58.923 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:59.924 7f22cb66ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:16:59.958 7f22cb61fc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:16:59.959 7f22cb61fc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:16:59.959 7f22cb61fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:17:00.063 7f22cb66f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:17:00.063 7f22d02c3c80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:17:00.063 7f22d02c3c80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:17:00.063 7f22cb61fa40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:17:00.063 7f22cb61fa40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:17:00.070 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:17:00.071 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:17:00.071 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:17:00.071 7f22d02c3c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:17:08.558 7fae199aa0c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:17:08.558 7fae199aa0c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:17:08.558 7fae199aa0c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:17:08.561 7fae199aa0c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:17:08.562 7fae14f222c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:17:08.562 7fae14f222c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:17:08.562 7fae14f227c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:17:08.562 7fae14f227c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:17:08.563 7fae14f227c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:17:08.563 7fae14f227c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:17:08.563 7fae14f227c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.563 7fae14f227c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:17:08.563 7fae14f227c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.563 7fae14f227c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.563 7fae14f227c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352228563205 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:17:08.563 7fae14f227c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.564 7fae14f227c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.564 7fae14f227c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:17:08.564 7fae14f227c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.564 7fae14f227c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.564 7fae14f227c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:08.564 7fae14f227c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.565 7fae14f22e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:17:08.565 7fae14f22e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:17:08.566 7fae14f22c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:17:08.567 7fae14ed2540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.567 7fae14ed2540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:17:08.567 7fae14ed2540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.567 7fae14ed2540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.567 7fae14ed2540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:08.567 7fae14ed2540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.621 7fae14f227c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:17:08.629 7fae14f227c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:17:08.630 7fae14e700e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.631 7fae14e700e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:08.631 7fae14e700e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.631 7fae14e700e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.631 7fae14e700e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:08.631 7fae14e700e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.631 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:08.633 7fae14ed27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.633 7fae14ed27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:08.633 7fae14ed27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.633 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:08.635 7fae14f227c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:17:08.635 7fae14ed27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.635 7fae14ed27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:08.635 7fae14ed27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.644 7fae14f227c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:17:08.645 7fae14e70c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.645 7fae14e70c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:08.645 7fae14e70c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.647 7fae14f227c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:17:08.647 7fae14e70c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.648 7fae14e70c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:08.648 7fae14e70c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.648 7fae14f227c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:17:08.648 7fae14f227c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:17:08.648 7fae14f227c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:17:08.648 7fae14e70f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:17:08.649 7fae14da7220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.649 7fae14da7220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:17:08.649 7fae14da7220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.649 7fae14da7220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.649 7fae14da7220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:17:08.649 7fae14da7220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.651 7fae14f227c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:17:08.655 7fae14e70f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:17:08.656 7fae14da7ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.656 7fae14da7ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:17:08.656 7fae14da7ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.674 7fae14f227c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:17:08.676 7fae14da7e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.676 7fae14da7e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:17:08.676 7fae14da7e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:08.695 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:08.696 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:08.793 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:08.793 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:08.928 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:08.928 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:09.134 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:09.134 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:09.565 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:09.596 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:09.597 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:10.274 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:10.275 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:10.566 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:11.568 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:11.662 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:11.663 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:12.564 7fae14da7e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:17:12.565 7fae14da7e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:17:12.565 7fae14da7e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:17:12.565 7fae14da7e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:17:12.565 7fae14da7e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:17:12.565 7fae14da7e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:17:12.565 7fae14da7e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:17:12.565 7fae14da7e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:17:12.567 7fae14da7e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:17:12.567 7fae14da7e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:17:12.567 7fae14da7e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:17:12.567 7fae14da7720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:17:12.568 7fae14da7720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:17:12.569 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:13.570 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:14.562 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:14.563 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:14.571 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:15.572 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:16.573 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:17.574 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:18.575 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:19.576 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:20.576 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:21.120 7fae14ed2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:21.120 7fae14ed2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:21.577 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:22.578 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:23.580 7fae14f22f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:23.631 7fae14ed2c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:17:23.631 7fae14ed2c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:17:23.632 7fae14ed2c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:17:23.772 7fae14f227c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:17:23.772 7fae199aa0c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:17:23.772 7fae199aa0c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:17:23.773 7fae14ed2b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:17:23.773 7fae14ed2b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:17:23.782 7fae199aa0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:17:32.064 7f72f747fa00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:17:32.064 7f72f747fa00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:17:32.064 7f72f747fa00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:17:32.067 7f72f747fa00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:17:32.067 7f72f24222c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:17:32.067 7f72f24222c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:17:32.068 7f72f24227c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:17:32.068 7f72f24227c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:17:32.068 7f72f24227c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:17:32.068 7f72f24227c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:17:32.069 7f72f24227c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.069 7f72f24227c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:17:32.069 7f72f24227c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.069 7f72f24227c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.069 7f72f24227c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352252068728 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:17:32.069 7f72f24227c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.069 7f72f24227c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.069 7f72f24227c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:17:32.070 7f72f24227c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.070 7f72f24227c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.070 7f72f24227c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:32.070 7f72f24227c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.070 7f72f2422e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:17:32.071 7f72f2422e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:17:32.071 7f72f2422c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:17:32.072 7f72f23d2540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.072 7f72f23d2540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:17:32.072 7f72f23d2540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.072 7f72f23d2540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.072 7f72f23d2540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:32.072 7f72f23d2540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.127 7f72f24227c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:17:32.135 7f72f24227c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:17:32.136 7f72f2376040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.136 7f72f2376040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:32.136 7f72f2376040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.136 7f72f2376040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.136 7f72f2376040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:32.136 7f72f2376040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.137 7f72f23d2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:32.137 7f72f23d2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:32.138 7f72f23d27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.138 7f72f23d27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:32.138 7f72f23d27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.139 7f72f24227c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:17:32.139 7f72f23d27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.140 7f72f23d27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:32.140 7f72f23d27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.149 7f72f24227c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:17:32.149 7f72f2376ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.149 7f72f2376ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:32.149 7f72f2376ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.165 7f72f24227c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:17:32.166 7f72f2376ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.166 7f72f2376ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:32.166 7f72f2376ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.166 7f72f24227c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:17:32.166 7f72f24227c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:17:32.166 7f72f24227c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:17:32.167 7f72f2376e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:17:32.167 7f72f229d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.167 7f72f229d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:17:32.167 7f72f229d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.167 7f72f229d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.168 7f72f229d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:17:32.168 7f72f229d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.170 7f72f24227c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:17:32.173 7f72f2376e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:17:32.173 7f72f229dae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.173 7f72f229dae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:17:32.173 7f72f229dae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.189 7f72f24227c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:17:32.190 7f72f229de00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.190 7f72f229de00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:17:32.190 7f72f229de00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:32.205 7f72f23d2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:32.206 7f72f23d2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:32.301 7f72f23d2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:32.301 7f72f23d2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:32.543 7f72f23d2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:32.544 7f72f23d2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:32.857 7f72f23d2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:32.857 7f72f23d2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:33.071 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:33.269 7f72f23d2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:33.270 7f72f23d2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:34.073 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:34.292 7f72f23d2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:34.293 7f72f23d2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:35.073 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:35.698 7f72f229de00 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:17:36.069 7f72f229de00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:17:36.070 7f72f229de00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:17:36.070 7f72f229de00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:17:36.071 7f72f229de00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:17:36.071 7f72f229de00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:17:36.071 7f72f229de00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:17:36.071 7f72f229de00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:17:36.071 7f72f229de00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:17:36.071 7f72f229de00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:17:36.072 7f72f229de00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:17:36.072 7f72f229de00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:17:36.072 7f72f229d720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:17:36.073 7f72f229d720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:17:36.074 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:37.076 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:37.122 7f72f23d2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:37.123 7f72f23d2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:38.077 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:39.079 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:40.080 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:41.081 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:42.082 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:43.083 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:44.084 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:44.908 7f72f23d2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:44.909 7f72f23d2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:45.085 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:46.086 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:47.087 7f72f2422f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:47.119 7f72f23d2a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:17:47.119 7f72f23d2a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:17:47.120 7f72f23d2a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:17:47.241 7f72f24227c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:17:47.242 7f72f747fa00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:17:47.242 7f72f747fa00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:17:47.242 7f72f23d29a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:17:47.242 7f72f23d29a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:17:47.249 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:17:47.250 7f72f747fa00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:17:55.750 7fdb70017ec0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:17:55.750 7fdb70017ec0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:17:55.750 7fdb70017ec0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:17:55.753 7fdb70017ec0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:17:55.754 7fdb6b58f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:17:55.754 7fdb6b58f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:17:55.754 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:17:55.754 7fdb6b58f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:17:55.755 7fdb6b58f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:17:55.755 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:17:55.755 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.755 7fdb6b58f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:17:55.755 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.755 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.755 7fdb6b58f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352275755311 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:17:55.756 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.756 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.756 7fdb6b58f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:17:55.756 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.756 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.757 7fdb6b58f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:55.757 7fdb6b58f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.757 7fdb6b58fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:17:55.757 7fdb6b58fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:17:55.758 7fdb6b58fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:17:55.759 7fdb6b53f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.759 7fdb6b53f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:17:55.759 7fdb6b53f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.759 7fdb6b53f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.759 7fdb6b53f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:55.759 7fdb6b53f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.815 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:17:55.823 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:17:55.824 7fdb6b4df040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.825 7fdb6b4df040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:55.825 7fdb6b4df040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.825 7fdb6b4df040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.825 7fdb6b4df040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:55.825 7fdb6b4df040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.826 7fdb6b53fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:55.827 7fdb6b53fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:55.828 7fdb6b53f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.828 7fdb6b53f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:55.828 7fdb6b53f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.829 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:17:55.830 7fdb6b53f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.830 7fdb6b53f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:55.830 7fdb6b53f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.844 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:17:55.845 7fdb6b4dfae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.845 7fdb6b4dfae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:17:55.845 7fdb6b4dfae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.848 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:17:55.849 7fdb6b4dfae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.849 7fdb6b4dfae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:17:55.849 7fdb6b4dfae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.849 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:17:55.849 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:17:55.850 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:17:55.850 7fdb6b4dfe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:17:55.851 7fdb6b4120e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.851 7fdb6b4120e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:17:55.851 7fdb6b4120e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.851 7fdb6b4120e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.851 7fdb6b4120e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:17:55.851 7fdb6b4120e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.855 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:17:55.859 7fdb6b4dfe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:17:55.859 7fdb6b4129a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.859 7fdb6b4129a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:17:55.859 7fdb6b4129a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.878 7fdb6b58f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:17:55.879 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.879 7fdb6b412cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:17:55.879 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:17:55.923 7fdb6b53fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:55.924 7fdb6b53fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:56.081 7fdb6b53fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:56.082 7fdb6b53fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:56.411 7fdb6b53fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:56.412 7fdb6b53fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:56.758 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:56.974 7fdb6b53fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:56.975 7fdb6b53fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:57.759 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:57.878 7fdb6b53fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:57.879 7fdb6b53fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:17:58.760 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:59.755 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:17:59.756 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:17:59.756 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:17:59.756 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:17:59.757 7fdb6b412cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:17:59.757 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:17:59.757 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:17:59.757 7fdb6b412cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:17:59.759 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:17:59.759 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:17:59.759 7fdb6b412cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:17:59.759 7fdb6b4125e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:17:59.760 7fdb6b4125e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:17:59.761 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:17:59.862 7fdb6b53fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:17:59.863 7fdb6b53fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:00.763 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:01.764 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:02.765 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:03.013 7fdb6b4125e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:18:03.767 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:03.987 7fdb6b53fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:03.987 7fdb6b53fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:04.768 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:05.769 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:06.770 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:07.771 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:08.772 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:09.774 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:10.775 7fdb6b58ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:10.819 7fdb6b53fa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:18:10.819 7fdb6b53fa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:18:10.820 7fdb6b53fa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:18:10.886 7fdb6b58f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:18:10.895 7fdb70017ec0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:18:10.895 7fdb70017ec0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:18:10.895 7fdb6b53f9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:18:10.895 7fdb6b53f9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:18:10.916 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:18:10.916 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:18:10.916 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:18:10.916 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:18:10.917 7fdb70017ec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:18:19.468 7f09b8ecf980 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:18:19.468 7f09b8ecf980 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:18:19.468 7f09b8ecf980 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:18:19.471 7f09b8ecf980 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:18:19.472 7f09b3e862c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:18:19.472 7f09b3e862c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:18:19.472 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:18:19.473 7f09b3e867c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:18:19.473 7f09b3e867c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:18:19.473 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:18:19.473 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.473 7f09b3e867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:18:19.473 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.473 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.474 7f09b3e867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352299473368 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:18:19.474 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.474 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.474 7f09b3e867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:18:19.474 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.474 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.475 7f09b3e867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:18:19.475 7f09b3e867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.475 7f09b3e86e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:18:19.475 7f09b3e86e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:18:19.476 7f09b3e86c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:18:19.477 7f09b3e35540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.477 7f09b3e35540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:18:19.477 7f09b3e35540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.477 7f09b3e35540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.477 7f09b3e35540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:18:19.477 7f09b3e35540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.531 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:18:19.538 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:18:19.539 7f09b3e35b80 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:19.540 7f09b3dc60e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.540 7f09b3dc60e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:18:19.540 7f09b3dc60e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.540 7f09b3dc60e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.540 7f09b3dc60e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:18:19.540 7f09b3dc60e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.541 7f09b3e35b80 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:19.542 7f09b3e357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.543 7f09b3e357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:18:19.543 7f09b3e357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.544 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:18:19.544 7f09b3e357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.544 7f09b3e357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:18:19.544 7f09b3e357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.555 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:18:19.556 7f09b3dc6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.556 7f09b3dc6c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:18:19.556 7f09b3dc6c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.563 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:18:19.564 7f09b3dc6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.564 7f09b3dc6c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:18:19.564 7f09b3dc6c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.564 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:18:19.564 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:18:19.565 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:18:19.565 7f09b3dc6f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:18:19.565 7f09b3777220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.566 7f09b3777220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:18:19.566 7f09b3777220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.566 7f09b3777220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.566 7f09b3777220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:18:19.566 7f09b3777220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.569 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:18:19.572 7f09b3dc6f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:18:19.572 7f09b3777ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.572 7f09b3777ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:18:19.572 7f09b3777ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.590 7f09b3e867c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:18:19.591 7f09b3777e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.591 7f09b3777e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:18:19.592 7f09b3777e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:19.667 7f09b3e35b80 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:19.668 7f09b3e35b80 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:19.953 7f09b3e35b80 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:19.954 7f09b3e35b80 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:20.344 7f09b3e35b80 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:20.345 7f09b3e35b80 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:20.476 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:21.024 7f09b3e35b80 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:21.024 7f09b3e35b80 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:21.477 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:22.114 7f09b3e35b80 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:22.115 7f09b3e35b80 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:22.478 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:23.474 7f09b3777e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:18:23.475 7f09b3777e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:18:23.475 7f09b3777e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:18:23.475 7f09b3777e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:18:23.476 7f09b3777e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:18:23.476 7f09b3777e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:18:23.476 7f09b3777e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:18:23.476 7f09b3777e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:18:23.478 7f09b3777e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:18:23.478 7f09b3777e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:18:23.478 7f09b3777e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:18:23.478 7f09b5f7e360 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:18:23.479 7f09b5f7e360 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:18:23.480 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:24.481 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:24.701 7f09b3e35b80 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:24.702 7f09b3e35b80 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:25.481 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:26.482 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:27.483 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:28.485 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:29.486 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:30.419 7f09b4628c20 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:18:30.487 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:30.501 7f09b3e35b80 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:30.502 7f09b3e35b80 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:31.488 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:32.489 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:33.490 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:34.492 7f09b3e86f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:34.521 7f09b3e35b80 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:18:34.521 7f09b3e35b80 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:18:34.522 7f09b3e35b80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:18:34.630 7f09b3e867c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:18:34.630 7f09b8ecf980 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:18:34.631 7f09b8ecf980 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:18:34.631 7f09b3e35a40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:18:34.631 7f09b3e35a40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:18:34.638 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:18:34.639 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:18:34.639 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:18:34.639 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:18:34.639 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:18:34.639 7f09b8ecf980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:18:43.201 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:18:43.201 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:18:43.201 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:18:43.205 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:18:43.205 7fdcafb9c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:18:43.205 7fdcafb9c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:18:43.206 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:18:43.206 7fdcafb9c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:18:43.206 7fdcafb9c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:18:43.206 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:18:43.207 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.207 7fdcafb9c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:18:43.207 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.207 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.207 7fdcafb9c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352323206701 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:18:43.207 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.207 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.208 7fdcafb9c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:18:43.208 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.208 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.208 7fdcafb9c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:18:43.208 7fdcafb9c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.208 7fdcafb9ce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:18:43.209 7fdcafb9ce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:18:43.210 7fdcafb9cc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:18:43.210 7fdcafb4b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.210 7fdcafb4b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:18:43.210 7fdcafb4b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.210 7fdcafb4b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.210 7fdcafb4b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:18:43.210 7fdcafb4b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.268 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:18:43.283 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:18:43.284 7fdcb22e7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.285 7fdcb22e7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:18:43.285 7fdcb22e7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.285 7fdcb22e7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.285 7fdcb22e7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:18:43.285 7fdcb22e7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.285 7fdcafb4ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:43.286 7fdcafb4ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:43.287 7fdcafb4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.287 7fdcafb4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:18:43.287 7fdcafb4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.288 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:18:43.289 7fdcafb4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.289 7fdcafb4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:18:43.289 7fdcafb4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.298 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:18:43.299 7fdcb22e7ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.299 7fdcb22e7ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:18:43.299 7fdcb22e7ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.301 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:18:43.302 7fdcb22e7ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.302 7fdcb22e7ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:18:43.302 7fdcb22e7ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.302 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:18:43.302 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:18:43.302 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:18:43.303 7fdcb22e7e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:18:43.303 7fdcafa200e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.303 7fdcafa200e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:18:43.303 7fdcafa200e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.303 7fdcafa200e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.303 7fdcafa200e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:18:43.303 7fdcafa200e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.306 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:18:43.309 7fdcb22e7e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:18:43.310 7fdcafa209a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.310 7fdcafa209a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:18:43.310 7fdcafa209a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.325 7fdcafb9c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:18:43.327 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.327 7fdcafa20cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:18:43.327 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:18:43.356 7fdcafb4ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:43.369 7fdcafb4ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:43.484 7fdcafb4ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:43.484 7fdcafb4ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:43.627 7fdcafb4ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:43.628 7fdcafb4ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:43.984 7fdcafb4ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:43.984 7fdcafb4ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:44.210 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:44.784 7fdcafb4ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:44.785 7fdcafb4ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:45.211 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:46.212 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:46.584 7fdcafb4ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:46.585 7fdcafb4ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:47.207 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:18:47.208 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:18:47.208 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:18:47.208 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:18:47.208 7fdcafa20cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:18:47.209 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:18:47.209 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:18:47.209 7fdcafa20cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:18:47.211 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:18:47.211 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:18:47.211 7fdcafa20cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:18:47.211 7fdcafa205e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:18:47.212 7fdcafa205e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:18:47.213 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:48.213 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:49.214 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:50.102 7fdcafb4ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:18:50.103 7fdcafb4ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:18:50.216 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:51.216 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:52.218 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:53.219 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:54.220 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:55.221 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:56.222 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:57.223 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:58.059 7fdcb033ec20 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:18:58.225 7fdcafb9cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:18:58.241 7fdcafb4ba40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:18:58.242 7fdcafb4ba40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:18:58.242 7fdcafb4ba40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:18:58.426 7fdcafb9c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:18:58.427 7fdcb4e510c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:18:58.427 7fdcb4e510c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:18:58.427 7fdcafb4b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:18:58.427 7fdcafb4b9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:18:58.434 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:18:58.435 7fdcb4e510c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:19:06.892 7f2b950cc680 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:19:06.892 7f2b950cc680 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:19:06.892 7f2b950cc680 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:19:06.895 7f2b950cc680 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:19:06.896 7f2b9064c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:19:06.896 7f2b9064c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:19:06.896 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:19:06.896 7f2b9064c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:19:06.897 7f2b9064c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:19:06.897 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:19:06.897 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.897 7f2b9064c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:19:06.897 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.897 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.897 7f2b9064c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352346897012 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:19:06.897 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.898 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.898 7f2b9064c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:19:06.898 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.898 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.898 7f2b9064c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:06.898 7f2b9064c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.899 7f2b9064ce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:19:06.899 7f2b9064ce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:19:06.900 7f2b9064cc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:19:06.900 7f2b905fc540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.901 7f2b905fc540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:19:06.901 7f2b905fc540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.901 7f2b905fc540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.901 7f2b905fc540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:06.901 7f2b905fc540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.954 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:19:06.961 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:19:06.962 7f2b91c31040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.963 7f2b91c31040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:06.963 7f2b91c31040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.963 7f2b91c31040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.963 7f2b91c31040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:06.963 7f2b91c31040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.964 7f2b905fca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:06.964 7f2b91c31040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.964 7f2b91c31040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:06.965 7f2b91c31040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.965 7f2b905fca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:06.967 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:19:06.967 7f2b905fc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.967 7f2b905fc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:06.967 7f2b905fc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.979 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:19:06.980 7f2b91c31ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.980 7f2b91c31ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:06.980 7f2b91c31ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.983 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:19:06.983 7f2b91c31ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.983 7f2b91c31ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:06.983 7f2b91c31ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.983 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:19:06.984 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:19:06.984 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:19:06.984 7f2b91c31e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:19:06.984 7f2b91cc20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.985 7f2b91cc20e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:19:06.985 7f2b91cc20e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.985 7f2b91cc20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.985 7f2b91cc20e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:19:06.985 7f2b91cc20e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.987 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:19:06.991 7f2b91c31e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:19:06.991 7f2b91cc29a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:06.991 7f2b91cc29a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:19:06.991 7f2b91cc29a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:07.007 7f2b9064c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:19:07.008 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:07.008 7f2b91cc2cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:19:07.008 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:07.095 7f2b905fca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:07.096 7f2b905fca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:07.414 7f2b905fca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:07.415 7f2b905fca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:07.900 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:08.005 7f2b905fca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:08.006 7f2b905fca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:08.901 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:09.628 7f2b905fca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:09.629 7f2b905fca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:09.902 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:10.897 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:19:10.898 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:19:10.898 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:19:10.898 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:19:10.898 7f2b91cc2cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:19:10.898 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:19:10.898 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:19:10.898 7f2b91cc2cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:19:10.900 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:19:10.900 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:19:10.900 7f2b91cc2cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:19:10.900 7f2b91cc25e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:19:10.901 7f2b91cc25e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:19:10.903 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:11.904 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:12.905 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:13.289 7f2b905fca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:13.290 7f2b905fca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:13.906 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:14.907 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:15.909 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:16.909 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:17.910 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:18.911 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:19.912 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:20.596 7f2b905fca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:20.597 7f2b905fca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:20.914 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:21.903 7f2b905fca40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:19:21.903 7f2b905fca40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:19:21.904 7f2b905fca40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:19:21.916 7f2b9064cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:22.086 7f2b9064c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:19:22.087 7f2b950cc680 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:19:22.087 7f2b950cc680 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:19:22.087 7f2b905fc9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:19:22.087 7f2b905fc9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:19:22.094 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:19:22.094 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:19:22.094 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:19:22.094 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:19:22.094 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:19:22.095 7f2b950cc680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:19:30.606 7fd393e8da40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:19:30.606 7fd393e8da40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:19:30.606 7fd393e8da40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:19:30.610 7fd393e8da40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:19:30.610 7fd38ee2a2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:19:30.611 7fd38ee2a2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:19:30.611 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:19:30.611 7fd38ee2a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:19:30.612 7fd38ee2a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:19:30.612 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:19:30.612 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.612 7fd38ee2a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:19:30.612 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.612 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.612 7fd38ee2a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352370612095 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:19:30.612 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.613 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.613 7fd38ee2a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:19:30.613 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.613 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.613 7fd38ee2a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:30.614 7fd38ee2a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.614 7fd38ee2ae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:19:30.614 7fd38ee2ae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:19:30.615 7fd38ee2ac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:19:30.616 7fd38edd9540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.616 7fd38edd9540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:19:30.616 7fd38edd9540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.616 7fd38edd9540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.617 7fd38edd9540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:30.617 7fd38edd9540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.674 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:19:30.682 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:19:30.683 7fd38ed7b0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.683 7fd38ed7b0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:30.684 7fd38ed7b0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.684 7fd38ed7b0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.684 7fd38ed7b0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:30.684 7fd38ed7b0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.684 7fd38edd9c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:30.685 7fd38edd9c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:30.686 7fd38edd97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.686 7fd38edd97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:30.686 7fd38edd97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.688 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:19:30.688 7fd38edd97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.688 7fd38edd97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:30.688 7fd38edd97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.698 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:19:30.698 7fd38ed7bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.699 7fd38ed7bc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:30.699 7fd38ed7bc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.702 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:19:30.702 7fd38ed7bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.702 7fd38ed7bc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:30.702 7fd38ed7bc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.702 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:19:30.703 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:19:30.703 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:19:30.703 7fd38ed7bf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:19:30.703 7fd38eca8220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.704 7fd38eca8220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:19:30.704 7fd38eca8220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.704 7fd38eca8220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.704 7fd38eca8220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:19:30.704 7fd38eca8220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.706 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:19:30.710 7fd38ed7bf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:19:30.711 7fd38eca8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.711 7fd38eca8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:19:30.711 7fd38eca8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.728 7fd38ee2a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:19:30.729 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.729 7fd38eca8e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:19:30.729 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:30.814 7fd38edd9c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:30.815 7fd38edd9c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:31.055 7fd38edd9c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:31.056 7fd38edd9c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:31.592 7fd38edd9c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:31.593 7fd38edd9c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:31.615 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:32.349 7fd38edd9c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:32.350 7fd38edd9c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:32.616 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:33.618 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:33.669 7fd38edd9c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:33.670 7fd38edd9c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:34.612 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:19:34.613 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:19:34.613 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:19:34.613 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:19:34.614 7fd38eca8e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:19:34.614 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:19:34.614 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:19:34.616 7fd38eca8e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:19:34.616 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:19:34.616 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:19:34.616 7fd38eca8e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:19:34.616 7fd38eca8720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:19:34.617 7fd38eca8720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:19:34.619 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:35.620 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:35.957 7fd38edd9c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:35.958 7fd38edd9c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:36.622 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:37.623 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:38.624 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:39.625 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:40.626 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:41.168 7fd38edd9c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:41.169 7fd38edd9c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:41.628 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:42.629 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:43.631 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:44.632 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:45.634 7fd38ee2af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:45.689 7fd38edd9c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:19:45.690 7fd38edd9c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:19:45.691 7fd38edd9c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:19:45.825 7fd38ee2a7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:19:45.825 7fd393e8da40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:19:45.825 7fd393e8da40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:19:45.825 7fd38edd9b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:19:45.825 7fd38edd9b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:19:45.833 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:19:45.833 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:19:45.834 7fd393e8da40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:19:54.198 7f87b77d9580 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:19:54.198 7f87b77d9580 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:19:54.198 7f87b77d9580 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:19:54.201 7f87b77d9580 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:19:54.202 7f87b2d592c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:19:54.202 7f87b2d592c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:19:54.202 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:19:54.203 7f87b2d597c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:19:54.203 7f87b2d597c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:19:54.203 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:19:54.203 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.203 7f87b2d597c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:19:54.203 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.203 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.204 7f87b2d597c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352394203450 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:19:54.204 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.204 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.204 7f87b2d597c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:19:54.204 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.204 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.205 7f87b2d597c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:54.205 7f87b2d597c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.205 7f87b2d59e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:19:54.205 7f87b2d59e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:19:54.206 7f87b2d59c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:19:54.206 7f87b2d09540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.207 7f87b2d09540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:19:54.207 7f87b2d09540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.207 7f87b2d09540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.207 7f87b2d09540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:54.207 7f87b2d09540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.261 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:19:54.269 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:19:54.270 7f87b2cf3040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.270 7f87b2cf3040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:54.270 7f87b2cf3040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.270 7f87b2cf3040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.270 7f87b2cf3040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:54.270 7f87b2cf3040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.271 7f87b2d09a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:54.272 7f87b2d09a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:54.272 7f87b2d097c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.273 7f87b2d097c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:54.273 7f87b2d097c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.274 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:19:54.274 7f87b2d097c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.274 7f87b2d097c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:54.274 7f87b2d097c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.284 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:19:54.284 7f87b2cf3ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.285 7f87b2cf3ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:19:54.285 7f87b2cf3ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.287 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:19:54.288 7f87b2cf3ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.288 7f87b2cf3ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:19:54.288 7f87b2cf3ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.288 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:19:54.289 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:19:54.289 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:19:54.289 7f87b2cf3e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:19:54.290 7f87b43330e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.290 7f87b43330e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:19:54.290 7f87b43330e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.290 7f87b43330e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.290 7f87b43330e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:19:54.290 7f87b43330e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.293 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:19:54.296 7f87b2cf3e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:19:54.296 7f87b43339a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.296 7f87b43339a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:19:54.296 7f87b43339a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.312 7f87b2d597c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:19:54.313 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.313 7f87b4333cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:19:54.313 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:19:54.383 7f87b2d09a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:54.383 7f87b2d09a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:54.662 7f87b2d09a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:54.663 7f87b2d09a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:55.054 7f87b2d09a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:55.055 7f87b2d09a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:55.207 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:56.121 7f87b2d09a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:56.122 7f87b2d09a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:56.208 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:57.209 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:58.204 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:19:58.205 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:19:58.205 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:19:58.205 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:19:58.205 7f87b4333cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:19:58.205 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:19:58.205 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:19:58.205 7f87b4333cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:19:58.207 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:19:58.207 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:19:58.207 7f87b4333cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:19:58.207 7f87b43335e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:19:58.208 7f87b43335e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:19:58.210 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:19:58.802 7f87b2d09a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:19:58.803 7f87b2d09a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:19:59.210 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:00.211 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:01.212 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:02.153 7f87b2d09a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:02.154 7f87b2d09a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:02.214 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:03.216 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:04.216 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:05.217 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:06.219 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:07.220 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:08.221 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:08.864 7f87b2d09a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:08.865 7f87b2d09a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:09.222 7f87b2d59f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:09.267 7f87b2d09a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:20:09.267 7f87b2d09a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:20:09.268 7f87b2d09a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:20:09.357 7f87b2d597c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:20:09.357 7f87b77d9580 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:20:09.357 7f87b77d9580 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:20:09.357 7f87b2d099a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:20:09.357 7f87b2d099a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:20:09.364 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:20:09.364 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:20:09.364 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:20:09.364 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:20:09.365 7f87b77d9580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:20:17.745 7f19f8e64c80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:20:17.745 7f19f8e64c80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:20:17.745 7f19f8e64c80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:20:17.748 7f19f8e64c80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:20:17.748 7f19f40992c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:20:17.749 7f19f40992c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:20:17.749 7f19f40997c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:20:17.749 7f19f40997c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:20:17.749 7f19f40997c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:20:17.750 7f19f40997c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:20:17.750 7f19f40997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.750 7f19f40997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:20:17.750 7f19f40997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.750 7f19f40997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.750 7f19f40997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352417749898 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:20:17.750 7f19f40997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.750 7f19f40997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.751 7f19f40997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:20:17.751 7f19f40997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.751 7f19f40997c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.751 7f19f40997c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:20:17.751 7f19f40997c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.751 7f19f4099e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:20:17.751 7f19f4099e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:20:17.752 7f19f4099c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:20:17.753 7f19f4048540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.753 7f19f4048540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:20:17.753 7f19f4048540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.753 7f19f4048540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.753 7f19f4048540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:20:17.753 7f19f4048540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.805 7f19f40997c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:20:17.812 7f19f40997c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:20:17.813 7f19f3fee0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.813 7f19f3fee0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:20:17.813 7f19f3fee0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.813 7f19f3fee0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.813 7f19f3fee0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:20:17.813 7f19f3fee0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.814 7f19f4048c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:17.815 7f19f4048c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:17.815 7f19f40487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.815 7f19f40487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:20:17.815 7f19f40487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.817 7f19f40997c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:20:17.817 7f19f40487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.817 7f19f40487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:20:17.817 7f19f40487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.826 7f19f40997c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:20:17.827 7f19f3feec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.827 7f19f3feec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:20:17.827 7f19f3feec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.831 7f19f40997c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:20:17.831 7f19f3feec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.831 7f19f3feec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:20:17.831 7f19f3feec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.831 7f19f40997c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:20:17.832 7f19f40997c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:20:17.832 7f19f40997c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:20:17.832 7f19f3feef40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:20:17.833 7f19f564b220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.833 7f19f564b220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:20:17.833 7f19f564b220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.833 7f19f564b220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.833 7f19f564b220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:20:17.833 7f19f564b220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.836 7f19f40997c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:20:17.840 7f19f3feef40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:20:17.840 7f19f564bae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.840 7f19f564bae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:20:17.840 7f19f564bae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.858 7f19f40997c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:20:17.860 7f19f564be00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.860 7f19f564be00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:20:17.860 7f19f564be00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:17.892 7f19f4048c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:17.892 7f19f4048c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:18.051 7f19f4048c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:18.051 7f19f4048c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:18.284 7f19f4048c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:18.285 7f19f4048c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:18.607 7f19f4048c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:18.609 7f19f4048c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:18.753 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:19.440 7f19f4048c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:19.441 7f19f4048c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:19.754 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:20.217 7f19f564be00 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:20:20.755 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:20.923 7f19f4048c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:20.924 7f19f4048c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:21.750 7f19f564be00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:20:21.751 7f19f564be00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:20:21.751 7f19f564be00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:20:21.751 7f19f564be00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:20:21.751 7f19f564be00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:20:21.751 7f19f564be00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:20:21.751 7f19f564be00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:20:21.751 7f19f564be00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:20:21.752 7f19f564be00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:20:21.752 7f19f564be00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:20:21.752 7f19f564be00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:20:21.752 7f19f564b860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:20:21.753 7f19f564b860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:20:21.756 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:22.758 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:23.759 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:24.038 7f19f4048c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:24.039 7f19f4048c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:24.761 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:25.762 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:26.763 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:27.764 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:28.394 7f19f4048c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:28.395 7f19f4048c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:28.765 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:29.767 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:30.768 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:31.769 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:32.769 7f19f4099f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:32.813 7f19f4048c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:20:32.814 7f19f4048c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:20:32.814 7f19f4048c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:20:32.907 7f19f40997c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:20:32.907 7f19f8e64c80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:20:32.908 7f19f8e64c80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:20:32.908 7f19f4048a40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:20:32.908 7f19f4048a40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:20:32.914 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:20:32.914 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:20:32.914 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:20:32.915 7f19f8e64c80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:20:41.229 7ff948c6b100 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:20:41.229 7ff948c6b100 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:20:41.229 7ff948c6b100 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:20:41.232 7ff948c6b100 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:20:41.233 7ff9441dc2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:20:41.233 7ff9441dc2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:20:41.233 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:20:41.233 7ff9441dc7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:20:41.234 7ff9441dc7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:20:41.234 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:20:41.234 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.235 7ff9441dc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:20:41.235 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.235 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.235 7ff9441dc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352441234647 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:20:41.235 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.235 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.236 7ff9441dc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:20:41.236 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.236 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.236 7ff9441dc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:20:41.236 7ff9441dc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.237 7ff9441dce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:20:41.237 7ff9441dce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:20:41.238 7ff9441dcc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:20:41.239 7ff94418c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.239 7ff94418c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:20:41.239 7ff94418c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.239 7ff94418c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.239 7ff94418c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:20:41.239 7ff94418c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.294 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:20:41.302 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:20:41.303 7ff9440ef0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.303 7ff9440ef0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:20:41.303 7ff9440ef0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.303 7ff9440ef0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.303 7ff9440ef0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:20:41.303 7ff9440ef0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.304 7ff94418cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:41.305 7ff94418cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:41.305 7ff94418c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.306 7ff94418c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:20:41.306 7ff94418c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.307 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:20:41.307 7ff94418c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.307 7ff94418c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:20:41.307 7ff94418c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.317 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:20:41.317 7ff9440efc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.317 7ff9440efc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:20:41.317 7ff9440efc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.320 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:20:41.320 7ff9440efc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.320 7ff9440efc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:20:41.320 7ff9440efc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.320 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:20:41.320 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:20:41.321 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:20:41.321 7ff9440eff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:20:41.321 7ff94405f220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.321 7ff94405f220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:20:41.321 7ff94405f220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.321 7ff94405f220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.322 7ff94405f220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:20:41.322 7ff94405f220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.324 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:20:41.327 7ff9440eff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:20:41.327 7ff94405fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.327 7ff94405fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:20:41.327 7ff94405fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.343 7ff9441dc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:20:41.344 7ff94405fe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.344 7ff94405fe00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:20:41.344 7ff94405fe00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:20:41.379 7ff94418cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:41.380 7ff94418cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:41.551 7ff94418cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:41.552 7ff94418cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:41.875 7ff94418cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:41.875 7ff94418cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:42.238 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:42.682 7ff94418cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:42.683 7ff94418cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:43.239 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:43.802 7ff94418cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:43.803 7ff94418cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:44.240 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:45.234 7ff94405fe00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:20:45.235 7ff94405fe00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:20:45.235 7ff94405fe00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:20:45.235 7ff94405fe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:20:45.235 7ff94405fe00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:20:45.235 7ff94405fe00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:20:45.235 7ff94405fe00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:20:45.236 7ff94405fe00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:20:45.237 7ff94405fe00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:20:45.237 7ff94405fe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:20:45.237 7ff94405fe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:20:45.237 7ff94405f720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:20:45.238 7ff94405f720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:20:45.242 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:46.243 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:46.640 7ff94418cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:46.641 7ff94418cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:47.244 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:47.793 7ff94418c9a0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:20:48.245 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:49.247 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:50.248 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:51.249 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:52.250 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:52.546 7ff94418cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:20:52.546 7ff94418cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:20:53.251 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:54.253 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:55.254 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:56.255 7ff9441dcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:20:56.262 7ff94418cc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:20:56.262 7ff94418cc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:20:56.263 7ff94418cc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:20:56.371 7ff9441dc7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:20:56.371 7ff948c6b100 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:20:56.371 7ff948c6b100 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:20:56.371 7ff94418cb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:20:56.371 7ff94418cb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:20:56.379 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:20:56.379 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:20:56.379 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:20:56.379 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:20:56.379 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:20:56.379 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:20:56.380 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:20:56.380 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:20:56.380 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:20:56.380 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:20:56.380 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:20:56.380 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:20:56.380 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:20:56.380 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:20:56.380 7ff948c6b100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:21:04.741 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:21:04.741 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:21:04.741 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:21:04.744 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:21:04.745 7fe3b1f882c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:21:04.745 7fe3b1f882c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:21:04.745 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:21:04.745 7fe3b1f887c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:21:04.746 7fe3b1f887c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:21:04.746 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:21:04.746 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.746 7fe3b1f887c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:21:04.746 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.746 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.746 7fe3b1f887c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352464746192 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:21:04.746 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.747 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.747 7fe3b1f887c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:21:04.747 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.747 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.747 7fe3b1f887c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:04.747 7fe3b1f887c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.748 7fe3b1f88e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:21:04.748 7fe3b1f88e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:21:04.749 7fe3b1f88c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:21:04.749 7fe3b1f37540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.749 7fe3b1f37540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:21:04.749 7fe3b1f37540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.750 7fe3b1f37540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.750 7fe3b1f37540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:04.750 7fe3b1f37540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.803 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:21:04.811 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:21:04.812 7fe3b46d5040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.812 7fe3b46d5040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:04.813 7fe3b46d5040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.813 7fe3b46d5040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.813 7fe3b46d5040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:04.813 7fe3b46d5040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.813 7fe3b1f37a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:04.814 7fe3b1f37a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:04.815 7fe3b1f377c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.815 7fe3b1f377c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:04.815 7fe3b1f377c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.816 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:21:04.817 7fe3b1f377c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.817 7fe3b1f377c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:04.817 7fe3b1f377c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.828 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:21:04.828 7fe3b46d5ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.829 7fe3b46d5ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:04.829 7fe3b46d5ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.831 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:21:04.832 7fe3b46d5ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.832 7fe3b46d5ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:04.832 7fe3b46d5ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.832 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:21:04.833 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:21:04.833 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:21:04.833 7fe3b46d5e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:21:04.834 7fe3b1e030e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.834 7fe3b1e030e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:21:04.834 7fe3b1e030e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.834 7fe3b1e030e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.834 7fe3b1e030e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:21:04.834 7fe3b1e030e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.837 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:21:04.840 7fe3b46d5e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:21:04.840 7fe3b1e039a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.841 7fe3b1e039a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:21:04.841 7fe3b1e039a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.857 7fe3b1f887c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:21:04.858 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.858 7fe3b1e03cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:21:04.858 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:04.914 7fe3b1f37a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:04.915 7fe3b1f37a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:05.162 7fe3b1f37a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:05.163 7fe3b1f37a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:05.737 7fe3b1f37a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:05.738 7fe3b1f37a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:05.749 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:06.750 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:07.271 7fe3b1f37a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:07.272 7fe3b1f37a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:07.751 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:08.746 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:21:08.747 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:21:08.747 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:21:08.748 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:21:08.748 7fe3b1e03cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:21:08.748 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:21:08.748 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:21:08.748 7fe3b1e03cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:21:08.750 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:21:08.750 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:21:08.750 7fe3b1e03cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:21:08.750 7fe3b1e035e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:21:08.751 7fe3b1e035e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:21:08.753 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:09.754 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:10.427 7fe3b1f37a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:10.427 7fe3b1f37a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:10.755 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:11.757 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:12.758 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:13.759 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:14.760 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:14.960 7fe3b1e035e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:21:15.675 7fe3b1f37a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:15.676 7fe3b1f37a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:15.762 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:16.763 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:17.764 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:18.765 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:19.766 7fe3b1f88f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:19.793 7fe3b1f37a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:21:19.793 7fe3b1f37a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:21:19.794 7fe3b1f37a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:21:19.947 7fe3b1f887c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:21:19.947 7fe3b6a0e880 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:21:19.947 7fe3b6a0e880 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:21:19.947 7fe3b1f379a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:21:19.947 7fe3b1f379a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:21:19.955 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:21:19.956 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:21:19.956 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:21:19.956 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:21:19.956 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:21:19.956 7fe3b6a0e880 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:21:28.207 7f3ff4759740 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:21:28.207 7f3ff4759740 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:21:28.207 7f3ff4759740 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:21:28.210 7f3ff4759740 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:21:28.211 7f3fefcd02c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:21:28.211 7f3fefcd02c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:21:28.211 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:21:28.211 7f3fefcd07c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:21:28.212 7f3fefcd07c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:21:28.212 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:21:28.212 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.212 7f3fefcd07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:21:28.212 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.212 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.212 7f3fefcd07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352488212016 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:21:28.212 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.212 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.213 7f3fefcd07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:21:28.213 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.213 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.213 7f3fefcd07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:28.213 7f3fefcd07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.213 7f3fefcd0e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:21:28.213 7f3fefcd0e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:21:28.214 7f3fefcd0c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:21:28.215 7f3fefc7f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.215 7f3fefc7f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:21:28.215 7f3fefc7f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.215 7f3fefc7f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.215 7f3fefc7f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:28.215 7f3fefc7f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.266 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:21:28.274 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:21:28.274 7f3fefbe5040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.275 7f3fefbe5040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:28.275 7f3fefbe5040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.275 7f3fefbe5040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.275 7f3fefbe5040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:28.275 7f3fefbe5040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.275 7f3fefc7fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:28.276 7f3fefc7fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:28.277 7f3fefc7f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.277 7f3fefc7f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:28.277 7f3fefc7f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.278 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:21:28.279 7f3fefc7f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.279 7f3fefc7f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:28.279 7f3fefc7f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.289 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:21:28.289 7f3fefbe5ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.290 7f3fefbe5ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:28.290 7f3fefbe5ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.292 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:21:28.292 7f3fefbe5ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.293 7f3fefbe5ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:28.293 7f3fefbe5ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.293 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:21:28.293 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:21:28.293 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:21:28.293 7f3fefbe5e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:21:28.294 7f3fefc5c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.294 7f3fefc5c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:21:28.294 7f3fefc5c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.294 7f3fefc5c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.294 7f3fefc5c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:21:28.294 7f3fefc5c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.297 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:21:28.300 7f3fefbe5e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:21:28.301 7f3fefc5c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.301 7f3fefc5c9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:21:28.301 7f3fefc5c9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.316 7f3fefcd07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:21:28.318 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.318 7f3fefc5ccc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:21:28.318 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:28.375 7f3fefc7fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:28.376 7f3fefc7fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:28.574 7f3fefc7fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:28.575 7f3fefc7fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:28.833 7f3fefc7fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:28.834 7f3fefc7fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:29.189 7f3fefc7fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:29.189 7f3fefc7fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:29.215 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:30.077 7f3fefc7fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:30.077 7f3fefc7fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:30.216 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:31.218 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:31.902 7f3fefc7fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:31.903 7f3fefc7fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:32.212 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:21:32.219 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:32.227 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:21:32.227 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:21:32.227 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:21:32.227 7f3fefc5ccc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:21:32.227 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:21:32.227 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:21:32.228 7f3fefc5ccc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:21:32.228 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:21:32.228 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:21:32.228 7f3fefc5ccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:21:32.228 7f3fefc5c720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:21:32.229 7f3fefc5c720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:21:33.221 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:34.222 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:35.224 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:35.606 7f3fefc7fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:35.607 7f3fefc7fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:36.225 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:37.226 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:38.227 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:39.228 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:40.229 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:41.230 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:42.231 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:42.708 7f3fefc5c720 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:21:43.232 7f3fefcd0f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:43.239 7f3fefc7fa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:21:43.239 7f3fefc7fa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:21:43.239 7f3fefc7fa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:21:43.350 7f3fefcd07c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:21:43.350 7f3ff4759740 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:21:43.350 7f3ff4759740 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:21:43.350 7f3fefc7f680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:21:43.350 7f3fefc7f680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:21:43.359 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:21:43.359 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:21:43.359 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:21:43.359 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:21:43.359 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:21:43.359 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:21:43.359 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:21:43.359 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:21:43.360 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:21:43.360 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:21:43.360 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:21:43.360 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:21:43.360 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:21:43.360 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:21:43.360 7f3ff4759740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:21:51.940 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:21:51.940 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:21:51.940 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:21:51.944 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:21:51.944 7f96932982c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:21:51.945 7f96932982c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:21:51.945 7f96932987c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:21:51.945 7f96932987c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:21:51.945 7f96932987c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:21:51.946 7f96932987c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:21:51.946 7f96932987c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.946 7f96932987c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:21:51.946 7f96932987c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.946 7f96932987c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.946 7f96932987c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352511945894 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:21:51.946 7f96932987c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.947 7f96932987c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.947 7f96932987c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:21:51.947 7f96932987c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.947 7f96932987c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.947 7f96932987c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:51.947 7f96932987c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.948 7f9693298e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:21:51.948 7f9693298e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:21:51.949 7f9693298c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:21:51.949 7f9693247540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.949 7f9693247540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:21:51.950 7f9693247540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.950 7f9693247540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:51.950 7f9693247540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:51.950 7f9693247540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.006 7f96932987c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:21:52.014 7f96932987c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:21:52.015 7f9695165040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.015 7f9695165040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:52.015 7f9695165040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.015 7f9695165040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.015 7f9695165040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:52.016 7f9695165040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.016 7f9693247a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:52.017 7f9693247a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:52.018 7f96932477c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.018 7f96932477c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:52.018 7f96932477c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.019 7f96932987c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:21:52.019 7f96932477c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.019 7f96932477c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:52.019 7f96932477c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.028 7f96932987c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:21:52.029 7f9695165ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.029 7f9695165ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:21:52.029 7f9695165ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.031 7f96932987c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:21:52.031 7f9695165ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.032 7f9695165ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:21:52.032 7f9695165ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.032 7f96932987c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:21:52.032 7f96932987c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:21:52.032 7f96932987c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:21:52.032 7f9695165e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:21:52.033 7f969311b0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.033 7f969311b0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:21:52.033 7f969311b0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.033 7f969311b0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.033 7f969311b0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:21:52.033 7f969311b0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.036 7f96932987c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:21:52.038 7f9695165e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:21:52.039 7f969311b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.039 7f969311b9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:21:52.039 7f969311b9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.054 7f96932987c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:21:52.055 7f969311bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.055 7f969311bcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:21:52.055 7f969311bcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:21:52.104 7f9693247a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:52.104 7f9693247a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:52.315 7f9693247a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:52.315 7f9693247a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:52.687 7f9693247a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:52.688 7f9693247a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:52.949 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:53.600 7f9693247a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:53.600 7f9693247a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:53.950 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:54.945 7f9693247a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:54.946 7f9693247a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:54.951 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:55.946 7f969311bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:21:55.947 7f969311bcc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:21:55.947 7f969311bcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:21:55.947 7f969311bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:21:55.947 7f969311bcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:21:55.947 7f969311bcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:21:55.947 7f969311bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:21:55.947 7f969311bcc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:21:55.949 7f969311bcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:21:55.949 7f969311bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:21:55.949 7f969311bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:21:55.949 7f969311b5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:21:55.950 7f969311b5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:21:55.952 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:56.953 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:57.954 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:58.559 7f9693247a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:21:58.560 7f9693247a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:21:58.955 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:21:59.957 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:00.958 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:01.959 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:02.961 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:03.962 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:04.963 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:05.965 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:06.965 7f9693298f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:06.995 7f9693247a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:22:06.995 7f9693247a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:22:06.996 7f9693247a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:22:07.102 7f96932987c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:22:07.103 7f9697d1a0c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:22:07.103 7f9697d1a0c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:22:07.103 7f96932479a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:22:07.103 7f96932479a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:22:07.110 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:22:07.111 7f9697d1a0c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:22:15.607 7f6e300318c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:22:15.608 7f6e300318c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:22:15.608 7f6e300318c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:22:15.611 7f6e300318c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:22:15.612 7f6e2b5bc2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:22:15.612 7f6e2b5bc2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:22:15.612 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:22:15.612 7f6e2b5bc7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:22:15.613 7f6e2b5bc7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:22:15.613 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:22:15.613 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.613 7f6e2b5bc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:22:15.613 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.613 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.613 7f6e2b5bc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352535613035 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:22:15.613 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.614 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.614 7f6e2b5bc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:22:15.614 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.614 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.614 7f6e2b5bc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:22:15.614 7f6e2b5bc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.615 7f6e2b5bce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:22:15.615 7f6e2b5bce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:22:15.616 7f6e2b5bcc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:22:15.616 7f6e2b56c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.616 7f6e2b56c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:22:15.616 7f6e2b56c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.617 7f6e2b56c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.617 7f6e2b56c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:22:15.617 7f6e2b56c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.675 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:22:15.683 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:22:15.684 7f6e2b50f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.684 7f6e2b50f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:22:15.684 7f6e2b50f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.684 7f6e2b50f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.684 7f6e2b50f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:22:15.684 7f6e2b50f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.685 7f6e2b56ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:15.686 7f6e2b56ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:15.686 7f6e2b56c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.687 7f6e2b56c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:22:15.687 7f6e2b56c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.688 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:22:15.688 7f6e2b56c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.688 7f6e2b56c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:22:15.688 7f6e2b56c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.697 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:22:15.698 7f6e2b50fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.698 7f6e2b50fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:22:15.698 7f6e2b50fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.700 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:22:15.700 7f6e2b50fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.700 7f6e2b50fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:22:15.700 7f6e2b50fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.701 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:22:15.701 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:22:15.701 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:22:15.701 7f6e2b50fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:22:15.702 7f6e2b4370e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.702 7f6e2b4370e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:22:15.702 7f6e2b4370e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.702 7f6e2b4370e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.702 7f6e2b4370e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:22:15.702 7f6e2b4370e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.704 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:22:15.707 7f6e2b50fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:22:15.707 7f6e2b4379a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.707 7f6e2b4379a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:22:15.708 7f6e2b4379a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.724 7f6e2b5bc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:22:15.725 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.725 7f6e2b437cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:22:15.726 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:15.801 7f6e2b56ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:15.802 7f6e2b56ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:15.968 7f6e2b56ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:15.969 7f6e2b56ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:16.338 7f6e2b56ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:16.339 7f6e2b56ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:16.616 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:17.238 7f6e2b56ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:17.239 7f6e2b56ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:17.617 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:18.618 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:19.258 7f6e2b56ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:19.259 7f6e2b56ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:19.613 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:22:19.614 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:22:19.614 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:22:19.615 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:22:19.615 7f6e2b437cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:22:19.615 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:22:19.615 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:22:19.615 7f6e2b437cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:22:19.617 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:22:19.617 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:22:19.617 7f6e2b437cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:22:19.617 7f6e2b4375e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:22:19.618 7f6e2b4375e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:22:19.619 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:20.620 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:21.622 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:22.340 7f6e2b56ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:22.341 7f6e2b56ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:22.623 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:23.624 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:24.626 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:25.627 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:26.628 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:27.630 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:28.079 7f6e2b56ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:28.080 7f6e2b56ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:28.631 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:29.633 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:30.633 7f6e2b5bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:30.691 7f6e2b56ca40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:22:30.691 7f6e2b56ca40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:22:30.692 7f6e2b56ca40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:22:30.814 7f6e2b5bc7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:22:30.815 7f6e300318c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:22:30.815 7f6e300318c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:22:30.815 7f6e2b56c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:22:30.815 7f6e2b56c9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:22:30.823 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:22:30.824 7f6e300318c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:22:39.245 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:22:39.245 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:22:39.245 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:22:39.248 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:22:39.249 7fa89766b2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:22:39.249 7fa89766b2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:22:39.249 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:22:39.250 7fa89766b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:22:39.250 7fa89766b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:22:39.250 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:22:39.250 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.250 7fa89766b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:22:39.250 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.250 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.250 7fa89766b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352559250357 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:22:39.251 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.251 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.251 7fa89766b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:22:39.251 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.251 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.251 7fa89766b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:22:39.252 7fa89766b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.252 7fa89766be00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:22:39.252 7fa89766be00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:22:39.253 7fa89766bc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:22:39.254 7fa89761b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.254 7fa89761b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:22:39.254 7fa89761b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.254 7fa89761b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.254 7fa89761b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:22:39.254 7fa89761b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.309 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:22:39.316 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:22:39.317 7fa8975bc040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.317 7fa8975bc040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:22:39.317 7fa8975bc040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.317 7fa8975bc040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.318 7fa8975bc040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:22:39.318 7fa8975bc040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.318 7fa89761ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:39.319 7fa89761ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:39.320 7fa89761b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.320 7fa89761b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:22:39.320 7fa89761b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.321 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:22:39.321 7fa89761b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.321 7fa89761b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:22:39.321 7fa89761b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.330 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:22:39.331 7fa8975bcae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.331 7fa8975bcae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:22:39.331 7fa8975bcae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.333 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:22:39.333 7fa8975bcae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.333 7fa8975bcae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:22:39.334 7fa8975bcae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.334 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:22:39.334 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:22:39.334 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:22:39.334 7fa8975bce00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:22:39.335 7fa8974ef0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.335 7fa8974ef0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:22:39.335 7fa8974ef0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.335 7fa8974ef0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.335 7fa8974ef0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:22:39.335 7fa8974ef0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.337 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:22:39.340 7fa8975bce00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:22:39.340 7fa8974ef9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.340 7fa8974ef9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:22:39.340 7fa8974ef9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.355 7fa89766b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:22:39.357 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.357 7fa8974efcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:22:39.357 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:22:39.436 7fa89761ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:39.436 7fa89761ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:39.615 7fa89761ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:39.616 7fa89761ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:39.873 7fa89761ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:39.874 7fa89761ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:40.254 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:40.385 7fa89761ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:40.386 7fa89761ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:41.255 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:41.791 7fa89761ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:41.791 7fa89761ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:42.256 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:43.251 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:22:43.252 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:22:43.252 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:22:43.252 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:22:43.252 7fa8974efcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:22:43.252 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:22:43.252 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:22:43.253 7fa8974efcc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:22:43.254 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:22:43.254 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:22:43.255 7fa8974efcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:22:43.255 7fa8974ef5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:22:43.256 7fa8974ef5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:22:43.257 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:44.259 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:44.755 7fa89761ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:44.755 7fa89761ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:45.260 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:46.261 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:47.263 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:48.264 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:49.265 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:50.266 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:50.556 7fa89761ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:22:50.557 7fa89761ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:22:51.267 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:52.268 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:53.269 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:54.270 7fa89766bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:22:54.273 7fa89761ba40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:22:54.273 7fa89761ba40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:22:54.274 7fa89761ba40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:22:54.434 7fa89766b7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:22:54.435 7fa89c0f6b40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:22:54.435 7fa89c0f6b40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:22:54.435 7fa89761b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:22:54.435 7fa89761b9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:22:54.442 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:22:54.442 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:22:54.442 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:22:54.442 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:22:54.442 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:22:54.442 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:22:54.442 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:22:54.443 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:22:54.443 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:22:54.443 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:22:54.443 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:22:54.443 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:22:54.443 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:22:54.443 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:22:54.443 7fa89c0f6b40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:23:02.967 7fd9051976c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:23:02.967 7fd9051976c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:23:02.967 7fd9051976c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:23:02.971 7fd9051976c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:23:02.971 7fd90052f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:23:02.971 7fd90052f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:23:02.971 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:23:02.972 7fd90052f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:23:02.972 7fd90052f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:23:02.972 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:23:02.972 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.973 7fd90052f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:23:02.973 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.973 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.973 7fd90052f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352582972563 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:23:02.973 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.974 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.974 7fd90052f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:23:02.974 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.974 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.974 7fd90052f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:02.974 7fd90052f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.975 7fd90052fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:23:02.975 7fd90052fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:23:02.976 7fd90052fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:23:02.976 7fd9004de540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.977 7fd9004de540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:23:02.977 7fd9004de540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.977 7fd9004de540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:02.977 7fd9004de540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:02.977 7fd9004de540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.033 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:23:03.040 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:23:03.041 7fd9004820e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.041 7fd9004820e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:03.041 7fd9004820e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.041 7fd9004820e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.041 7fd9004820e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:03.041 7fd9004820e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.042 7fd9004dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:03.043 7fd9004dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:03.043 7fd9004de7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.044 7fd9004de7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:03.044 7fd9004de7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.045 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:23:03.045 7fd9004de7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.045 7fd9004de7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:03.045 7fd9004de7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.054 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:23:03.055 7fd900482c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.055 7fd900482c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:03.055 7fd900482c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.057 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:23:03.057 7fd900482c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.057 7fd900482c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:03.057 7fd900482c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.057 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:23:03.058 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:23:03.058 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:23:03.058 7fd900482f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:23:03.058 7fd9003b3220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.059 7fd9003b3220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:23:03.059 7fd9003b3220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.059 7fd9003b3220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.059 7fd9003b3220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:23:03.059 7fd9003b3220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.061 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:23:03.064 7fd900482f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:23:03.064 7fd9003b3ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.064 7fd9003b3ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:23:03.064 7fd9003b3ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.079 7fd90052f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:23:03.080 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.081 7fd9003b3e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:23:03.081 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:03.115 7fd9004dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:03.116 7fd9004dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:03.288 7fd9004dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:03.288 7fd9004dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:03.523 7fd9004dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:03.524 7fd9004dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:03.976 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:04.043 7fd9004dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:04.044 7fd9004dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:04.832 7fd9003b3e00 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:23:04.935 7fd9004dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:04.935 7fd9004dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:04.977 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:05.978 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:06.232 7fd9004dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:06.233 7fd9004dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:06.972 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:23:06.973 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:23:06.973 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:23:06.974 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:23:06.974 7fd9003b3e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:23:06.974 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:23:06.974 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:23:06.974 7fd9003b3e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:23:06.974 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:23:06.974 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:23:06.974 7fd9003b3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:23:06.975 7fd9003b3860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:23:06.975 7fd9003b3860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:23:06.980 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:07.981 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:08.982 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:09.442 7fd9004dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:09.443 7fd9004dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:09.983 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:10.984 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:11.985 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:12.986 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:13.987 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:14.179 7fd9004dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:14.180 7fd9004dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:14.988 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:15.989 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:16.990 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:17.991 7fd90052ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:17.996 7fd9004dec20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:23:17.996 7fd9004dec20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:23:17.997 7fd9004dec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:23:18.129 7fd90052f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:23:18.130 7fd9051976c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:23:18.130 7fd9051976c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:23:18.130 7fd9004deb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:23:18.130 7fd9004deb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:23:18.137 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:23:18.137 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:23:18.138 7fd9051976c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:23:26.451 7f9a24c00a40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:23:26.451 7f9a24c00a40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:23:26.451 7f9a24c00a40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:23:26.454 7f9a24c00a40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:23:26.455 7f9a201812c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:23:26.455 7f9a201812c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:23:26.455 7f9a201817c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:23:26.455 7f9a201817c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:23:26.456 7f9a201817c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:23:26.456 7f9a201817c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:23:26.456 7f9a201817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.456 7f9a201817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:23:26.456 7f9a201817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.456 7f9a201817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.456 7f9a201817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352606456017 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:23:26.456 7f9a201817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.457 7f9a201817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.457 7f9a201817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:23:26.458 7f9a201817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.458 7f9a201817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.458 7f9a201817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:26.458 7f9a201817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.458 7f9a20181e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:23:26.458 7f9a20181e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:23:26.459 7f9a20181c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:23:26.460 7f9a20130540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.460 7f9a20130540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:23:26.460 7f9a20130540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.460 7f9a20130540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.460 7f9a20130540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:26.460 7f9a20130540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.515 7f9a201817c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:23:26.522 7f9a201817c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:23:26.523 7f9a200d4040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.523 7f9a200d4040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:26.523 7f9a200d4040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.523 7f9a200d4040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.524 7f9a200d4040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:26.524 7f9a200d4040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.524 7f9a20130a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:26.525 7f9a20130a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:26.525 7f9a201307c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.526 7f9a201307c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:26.526 7f9a201307c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.527 7f9a201817c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:23:26.527 7f9a201307c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.527 7f9a201307c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:26.527 7f9a201307c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.537 7f9a201817c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:23:26.537 7f9a200d4ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.537 7f9a200d4ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:26.537 7f9a200d4ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.539 7f9a201817c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:23:26.540 7f9a200d4ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.540 7f9a200d4ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:26.540 7f9a200d4ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.540 7f9a201817c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:23:26.540 7f9a201817c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:23:26.540 7f9a201817c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:23:26.541 7f9a200d4e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:23:26.541 7f9a200040e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.541 7f9a200040e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:23:26.541 7f9a200040e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.541 7f9a200040e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.541 7f9a200040e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:23:26.541 7f9a200040e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.544 7f9a201817c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:23:26.547 7f9a200d4e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:23:26.547 7f9a200049a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.548 7f9a200049a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:23:26.548 7f9a200049a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.562 7f9a201817c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:23:26.564 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.564 7f9a20004cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:23:26.564 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:26.608 7f9a20130a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:26.609 7f9a20130a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:26.760 7f9a20130a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:26.761 7f9a20130a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:27.042 7f9a20130a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:27.043 7f9a20130a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:27.460 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:27.802 7f9a20130a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:27.803 7f9a20130a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:28.461 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:29.350 7f9a20130a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:29.351 7f9a20130a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:29.462 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:30.456 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:23:30.457 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:23:30.458 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:23:30.458 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:23:30.458 7f9a20004cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:23:30.458 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:23:30.458 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:23:30.458 7f9a20004cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:23:30.460 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:23:30.460 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:23:30.460 7f9a20004cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:23:30.460 7f9a200045e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:23:30.461 7f9a200045e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:23:30.463 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:31.464 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:32.262 7f9a200045e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:23:32.465 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:32.549 7f9a20130a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:32.550 7f9a20130a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:33.467 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:34.468 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:35.469 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:36.470 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:37.471 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:38.472 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:39.474 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:40.474 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:40.994 7f9a20130a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:40.995 7f9a20130a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:41.476 7f9a20181f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:41.497 7f9a20130a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:23:41.498 7f9a20130a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:23:41.498 7f9a20130a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:23:41.625 7f9a201817c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:23:41.626 7f9a24c00a40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:23:41.626 7f9a24c00a40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:23:41.626 7f9a201309a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:23:41.626 7f9a201309a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:23:41.634 7f9a24c00a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:23:50.211 7fcc706a1940 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:23:50.211 7fcc706a1940 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:23:50.211 7fcc706a1940 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:23:50.214 7fcc706a1940 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:23:50.214 7fcc6bc1a2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:23:50.215 7fcc6bc1a2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:23:50.215 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:23:50.215 7fcc6bc1a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:23:50.215 7fcc6bc1a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:23:50.216 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:23:50.216 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.216 7fcc6bc1a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:23:50.216 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.216 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.216 7fcc6bc1a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352630215866 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:23:50.216 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.216 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.217 7fcc6bc1a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:23:50.217 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.217 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.217 7fcc6bc1a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:50.217 7fcc6bc1a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.217 7fcc6bc1ae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:23:50.218 7fcc6bc1ae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:23:50.219 7fcc6bc1ac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:23:50.219 7fcc6bbc9540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.219 7fcc6bbc9540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:23:50.219 7fcc6bbc9540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.219 7fcc6bbc9540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.220 7fcc6bbc9540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:50.220 7fcc6bbc9540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.273 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:23:50.281 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:23:50.282 7fcc6bb69040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.282 7fcc6bb69040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:50.282 7fcc6bb69040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.282 7fcc6bb69040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.282 7fcc6bb69040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:50.282 7fcc6bb69040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.283 7fcc6bbc9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:50.283 7fcc6bbc9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:50.284 7fcc6bbc97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.284 7fcc6bbc97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:50.284 7fcc6bbc97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.285 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:23:50.286 7fcc6bbc97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.286 7fcc6bbc97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:50.286 7fcc6bbc97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.295 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:23:50.295 7fcc6bb69ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.296 7fcc6bb69ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:23:50.296 7fcc6bb69ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.298 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:23:50.298 7fcc6bb69ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.298 7fcc6bb69ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:23:50.298 7fcc6bb69ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.298 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:23:50.298 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:23:50.299 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:23:50.299 7fcc6bb69e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:23:50.299 7fcc6ba9e0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.299 7fcc6ba9e0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:23:50.299 7fcc6ba9e0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.300 7fcc6ba9e0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.300 7fcc6ba9e0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:23:50.300 7fcc6ba9e0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.302 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:23:50.305 7fcc6bb69e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:23:50.305 7fcc6ba9e9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.305 7fcc6ba9e9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:23:50.305 7fcc6ba9e9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.320 7fcc6bc1a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:23:50.321 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.321 7fcc6ba9ecc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:23:50.321 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:23:50.351 7fcc6bbc9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:50.351 7fcc6bbc9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:50.519 7fcc6bbc9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:50.520 7fcc6bbc9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:50.864 7fcc6bbc9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:50.864 7fcc6bbc9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:51.219 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:51.811 7fcc6bbc9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:51.812 7fcc6bbc9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:52.220 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:53.222 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:53.486 7fcc6bbc9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:53.487 7fcc6bbc9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:54.215 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:23:54.216 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:23:54.216 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:23:54.216 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:23:54.217 7fcc6ba9ecc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:23:54.217 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:23:54.217 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:23:54.217 7fcc6ba9ecc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:23:54.218 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:23:54.219 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:23:54.219 7fcc6ba9ecc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:23:54.219 7fcc6ba9e5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:23:54.220 7fcc6ba9e5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:23:54.223 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:55.224 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:56.225 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:57.226 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:57.850 7fcc6bbc9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:23:57.850 7fcc6bbc9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:23:58.228 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:59.229 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:23:59.475 7fcc6ba9e5e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:24:00.231 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:01.233 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:02.234 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:03.235 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:04.236 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:05.238 7fcc6bc1af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:05.282 7fcc6bbc9a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:24:05.282 7fcc6bbc9a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:24:05.283 7fcc6bbc9a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:24:05.376 7fcc6bc1a7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:24:05.377 7fcc706a1940 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:24:05.377 7fcc706a1940 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:24:05.377 7fcc6bbc99a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:24:05.377 7fcc6bbc99a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:24:05.385 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:24:05.385 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:24:05.385 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:24:05.385 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:24:05.385 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:24:05.385 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:24:05.385 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:24:05.385 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:24:05.386 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:24:05.386 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:24:05.386 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:24:05.386 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:24:05.386 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:24:05.386 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:24:05.386 7fcc706a1940 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:24:13.667 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:24:13.667 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:24:13.667 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:24:13.670 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:24:13.671 7f6a97f4b2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:24:13.671 7f6a97f4b2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:24:13.671 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:24:13.672 7f6a97f4b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:24:13.672 7f6a97f4b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:24:13.672 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:24:13.672 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.672 7f6a97f4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:24:13.672 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.672 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.672 7f6a97f4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352653672269 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:24:13.673 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.673 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.673 7f6a97f4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:24:13.673 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.673 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.673 7f6a97f4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:24:13.673 7f6a97f4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.674 7f6a97f4be00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:24:13.674 7f6a97f4be00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:24:13.675 7f6a97f4bc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:24:13.675 7f6a97efb540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.676 7f6a97efb540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:24:13.676 7f6a97efb540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.676 7f6a97efb540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.676 7f6a97efb540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:24:13.676 7f6a97efb540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.735 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:24:13.743 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:24:13.744 7f6a97e5d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.744 7f6a97e5d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:24:13.744 7f6a97e5d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.744 7f6a97e5d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.744 7f6a97e5d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:24:13.745 7f6a97e5d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.745 7f6a97efbc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:13.746 7f6a97efbc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:13.747 7f6a97efb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.747 7f6a97efb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:24:13.747 7f6a97efb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.749 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:24:13.749 7f6a97efb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.749 7f6a97efb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:24:13.749 7f6a97efb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.761 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:24:13.762 7f6a97e5dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.762 7f6a97e5dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:24:13.762 7f6a97e5dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.765 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:24:13.765 7f6a97e5dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.765 7f6a97e5dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:24:13.765 7f6a97e5dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.766 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:24:13.766 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:24:13.766 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:24:13.766 7f6a97e5df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:24:13.767 7f6a97dd0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.767 7f6a97dd0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:24:13.767 7f6a97dd0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.768 7f6a97dd0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.768 7f6a97dd0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:24:13.768 7f6a97dd0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.771 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:24:13.776 7f6a97e5df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:24:13.776 7f6a97dd0ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.776 7f6a97dd0ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:24:13.776 7f6a97dd0ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.793 7f6a97f4b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:24:13.795 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.795 7f6a97dd0e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:24:13.795 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:13.830 7f6a97efbc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:13.831 7f6a97efbc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:14.062 7f6a97efbc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:14.063 7f6a97efbc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:14.430 7f6a97efbc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:14.447 7f6a97efbc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:14.675 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:15.342 7f6a97efbc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:15.343 7f6a97efbc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:15.676 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:16.676 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:16.850 7f6a97efbc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:16.851 7f6a97efbc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:17.672 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:24:17.677 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:17.687 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:24:17.687 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:24:17.687 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:24:17.687 7f6a97dd0e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:24:17.688 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:24:17.688 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:24:17.688 7f6a97dd0e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:24:17.688 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:24:17.688 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:24:17.688 7f6a97dd0e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:24:17.688 7f6a97dd0860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:24:17.689 7f6a97dd0860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:24:18.658 7f6a97efbc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:18.659 7f6a97efbc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:18.680 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:19.681 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:20.682 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:21.684 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:22.685 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:23.360 7f6a97efbc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:23.361 7f6a97efbc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:23.686 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:24.688 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:25.689 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:26.690 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:27.079 7f6a986edc20 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:24:27.692 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:28.685 7f6a97efbc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:24:28.685 7f6a97efbc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:24:28.686 7f6a97efbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:24:28.693 7f6a97f4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:28.841 7f6a97f4b7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:24:28.841 7f6a9c9c9c00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:24:28.841 7f6a9c9c9c00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:24:28.841 7f6a97efba40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:24:28.841 7f6a97efba40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:24:28.851 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:24:28.852 7f6a9c9c9c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:24:37.311 7fe2d034fd80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:24:37.312 7fe2d034fd80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:24:37.312 7fe2d034fd80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:24:37.315 7fe2d034fd80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:24:37.315 7fe2cb67c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:24:37.316 7fe2cb67c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:24:37.316 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:24:37.316 7fe2cb67c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:24:37.317 7fe2cb67c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:24:37.317 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:24:37.317 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.317 7fe2cb67c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:24:37.317 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.317 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.318 7fe2cb67c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352677317254 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:24:37.318 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.318 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.318 7fe2cb67c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:24:37.318 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.318 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.319 7fe2cb67c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:24:37.319 7fe2cb67c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.319 7fe2cb67ce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:24:37.319 7fe2cb67ce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:24:37.320 7fe2cb67cc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:24:37.321 7fe2cb62c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.321 7fe2cb62c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:24:37.321 7fe2cb62c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.321 7fe2cb62c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.321 7fe2cb62c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:24:37.321 7fe2cb62c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.377 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:24:37.385 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:24:37.386 7fe2cb5cd040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.386 7fe2cb5cd040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:24:37.386 7fe2cb5cd040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.386 7fe2cb5cd040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.387 7fe2cb5cd040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:24:37.387 7fe2cb5cd040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.387 7fe2cb62ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:37.388 7fe2cb62ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:37.389 7fe2cb62c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.389 7fe2cb62c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:24:37.389 7fe2cb62c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.390 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:24:37.390 7fe2cb62c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.390 7fe2cb62c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:24:37.390 7fe2cb62c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.400 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:24:37.400 7fe2cb5cdae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.401 7fe2cb5cdae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:24:37.401 7fe2cb5cdae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.403 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:24:37.403 7fe2cb5cdae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.403 7fe2cb5cdae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:24:37.403 7fe2cb5cdae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.404 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:24:37.404 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:24:37.404 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:24:37.404 7fe2cb5cde00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:24:37.405 7fe2cb4ff0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.405 7fe2cb4ff0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:24:37.405 7fe2cb4ff0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.405 7fe2cb4ff0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.405 7fe2cb4ff0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:24:37.405 7fe2cb4ff0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.408 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:24:37.411 7fe2cb5cde00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:24:37.412 7fe2cb4ff9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.412 7fe2cb4ff9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:24:37.412 7fe2cb4ff9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.427 7fe2cb67c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:24:37.428 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.429 7fe2cb4ffcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:24:37.429 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:24:37.515 7fe2cb62ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:37.516 7fe2cb62ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:37.722 7fe2cb62ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:37.722 7fe2cb62ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:38.138 7fe2cb62ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:38.139 7fe2cb62ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:38.320 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:38.758 7fe2cb62ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:38.758 7fe2cb62ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:39.322 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:39.640 7fe2cb62ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:39.640 7fe2cb62ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:40.323 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:41.317 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:24:41.319 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:24:41.319 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:24:41.319 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:24:41.319 7fe2cb4ffcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:24:41.319 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:24:41.319 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:24:41.320 7fe2cb4ffcc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:24:41.321 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:24:41.321 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:24:41.321 7fe2cb4ffcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:24:41.322 7fe2cb4ff5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:24:41.323 7fe2cb4ff5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:24:41.324 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:41.613 7fe2cb62ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:41.614 7fe2cb62ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:42.326 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:43.328 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:44.329 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:45.277 7fe2cb62ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:24:45.278 7fe2cb62ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:24:45.331 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:46.332 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:47.333 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:48.334 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:49.335 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:50.337 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:51.338 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:52.339 7fe2cb67cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:24:52.410 7fe2cb62ca40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:24:52.410 7fe2cb62ca40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:24:52.411 7fe2cb62ca40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:24:52.436 7fe2cb67c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:24:52.436 7fe2d034fd80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:24:52.436 7fe2d034fd80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:24:52.436 7fe2cb62c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:24:52.437 7fe2cb62c9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:24:52.443 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:24:52.444 7fe2d034fd80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:25:00.679 7f87c3d01d80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:25:00.679 7f87c3d01d80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:25:00.679 7f87c3d01d80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:25:00.682 7f87c3d01d80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:25:00.682 7f87bef322c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:25:00.683 7f87bef322c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:25:00.683 7f87bef327c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:25:00.683 7f87bef327c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:25:00.683 7f87bef327c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:25:00.684 7f87bef327c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:25:00.684 7f87bef327c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.684 7f87bef327c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:25:00.684 7f87bef327c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.684 7f87bef327c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.684 7f87bef327c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352700683825 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:25:00.685 7f87bef327c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.685 7f87bef327c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.685 7f87bef327c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:25:00.685 7f87bef327c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.685 7f87bef327c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.685 7f87bef327c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:00.685 7f87bef327c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.686 7f87bef32f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:25:00.686 7f87bef32f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:25:00.687 7f87bef32e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:25:00.687 7f87beee27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.687 7f87beee27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:25:00.687 7f87beee27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.687 7f87beee27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.687 7f87beee27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:00.687 7f87beee27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.745 7f87bef327c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:25:00.752 7f87bef327c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:25:00.753 7f87beee2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:00.753 7f87bee5d220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.753 7f87bee5d220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:00.753 7f87bee5d220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.753 7f87bee5d220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.753 7f87bee5d220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:00.753 7f87bee5d220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.754 7f87beee2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:00.755 7f87beee29a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.755 7f87beee29a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:00.756 7f87beee29a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.756 7f87bef327c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:25:00.757 7f87beee29a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.757 7f87beee29a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:00.757 7f87beee29a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.768 7f87bef327c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:25:00.768 7f87bee5dd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.768 7f87bee5dd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:00.769 7f87bee5dd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.771 7f87bef327c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:25:00.774 7f87bee5dd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.774 7f87bee5dd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:00.774 7f87bee5dd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.775 7f87bef327c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:25:00.775 7f87bef327c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:25:00.775 7f87bef327c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:25:00.775 7f87bee5dd60 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:25:00.776 7f87bedb6360 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.776 7f87bedb6360 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:25:00.776 7f87bedb6360 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.776 7f87bedb6360 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.776 7f87bedb6360 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:25:00.776 7f87bedb6360 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.779 7f87bef327c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:25:00.783 7f87bee5dd60 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:25:00.784 7f87bedb6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.784 7f87bedb6c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:25:00.784 7f87bedb6c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.804 7f87bef327c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:25:00.805 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.805 7f87bedb6f40 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:25:00.805 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:00.845 7f87beee2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:00.846 7f87beee2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:01.020 7f87beee2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:01.021 7f87beee2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:01.310 7f87beee2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:01.310 7f87beee2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:01.687 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:02.025 7f87beee2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:02.026 7f87beee2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:02.689 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:03.306 7f87beee2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:03.307 7f87beee2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:03.690 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:04.685 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:25:04.686 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:25:04.686 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:25:04.686 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:25:04.686 7f87bedb6f40 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:25:04.686 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:25:04.686 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:25:04.686 7f87bedb6f40 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:25:04.688 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:25:04.688 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:25:04.688 7f87bedb6f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:25:04.688 7f87c160a4a0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:25:04.689 7f87c160a4a0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:25:04.692 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:05.659 7f87beee2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:05.660 7f87beee2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:05.692 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:06.694 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:07.695 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:08.696 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:09.698 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:10.698 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:11.565 7f87beee2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:11.565 7f87beee2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:11.700 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:12.702 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:13.703 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:14.704 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:15.705 7f87beee22c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:15.782 7f87beee2c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:25:15.782 7f87beee2c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:25:15.783 7f87beee2c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:25:15.873 7f87bef327c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:25:15.874 7f87c3d01d80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:25:15.874 7f87c3d01d80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:25:15.874 7f87beee2b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:25:15.874 7f87beee2b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:25:15.881 7f87c3d01d80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:25:24.351 7fa7be3ca740 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:25:24.351 7fa7be3ca740 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:25:24.351 7fa7be3ca740 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:25:24.354 7fa7be3ca740 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:25:24.355 7fa7b93ab2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:25:24.355 7fa7b93ab2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:25:24.355 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:25:24.355 7fa7b93ab7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:25:24.356 7fa7b93ab7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:25:24.356 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:25:24.356 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.356 7fa7b93ab7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:25:24.356 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.356 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.357 7fa7b93ab7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352724356282 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:25:24.357 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.357 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.357 7fa7b93ab7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:25:24.357 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.357 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.357 7fa7b93ab7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:24.358 7fa7b93ab7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.358 7fa7b93abe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:25:24.358 7fa7b93abe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:25:24.359 7fa7b93abc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:25:24.359 7fa7b935b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.360 7fa7b935b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:25:24.360 7fa7b935b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.360 7fa7b935b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.360 7fa7b935b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:24.360 7fa7b935b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.413 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:25:24.420 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:25:24.421 7fa7b92ff040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.422 7fa7b92ff040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:24.422 7fa7b92ff040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.422 7fa7b92ff040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.422 7fa7b92ff040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:24.422 7fa7b92ff040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.422 7fa7b935ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:24.423 7fa7b935ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:24.424 7fa7b935b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.424 7fa7b935b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:24.424 7fa7b935b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.425 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:25:24.425 7fa7b935b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.426 7fa7b935b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:24.426 7fa7b935b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.438 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:25:24.438 7fa7b92ffae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.439 7fa7b92ffae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:24.439 7fa7b92ffae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.441 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:25:24.441 7fa7b92ffae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.441 7fa7b92ffae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:24.441 7fa7b92ffae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.442 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:25:24.442 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:25:24.442 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:25:24.442 7fa7b92ffe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:25:24.443 7fa7b922f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.443 7fa7b922f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:25:24.443 7fa7b922f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.443 7fa7b922f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.443 7fa7b922f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:25:24.443 7fa7b922f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.446 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:25:24.449 7fa7b92ffe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:25:24.449 7fa7b922f9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.449 7fa7b922f9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:25:24.450 7fa7b922f9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.465 7fa7b93ab7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:25:24.467 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.467 7fa7b922fcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:25:24.467 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:24.543 7fa7b935ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:24.544 7fa7b935ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:24.803 7fa7b935ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:24.804 7fa7b935ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:25.192 7fa7b935ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:25.193 7fa7b935ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:25.358 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:26.125 7fa7b935ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:26.125 7fa7b935ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:26.360 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:27.360 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:27.852 7fa7b935ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:27.852 7fa7b935ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:28.356 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:25:28.357 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:25:28.358 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:25:28.358 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:25:28.358 7fa7b922fcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:25:28.358 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:25:28.358 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:25:28.358 7fa7b922fcc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:25:28.360 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:25:28.360 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:25:28.360 7fa7b922fcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:25:28.360 7fa7b922f5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:25:28.361 7fa7b922f5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:25:28.361 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:29.362 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:30.363 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:31.365 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:32.366 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:32.664 7fa7b935ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:32.665 7fa7b935ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:33.367 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:34.369 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:35.370 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:36.372 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:37.373 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:38.373 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:39.375 7fa7b93abf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:39.396 7fa7b935ba40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:25:39.396 7fa7b935ba40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:25:39.397 7fa7b935ba40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:25:39.494 7fa7b93ab7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:25:39.495 7fa7be3ca740 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:25:39.495 7fa7be3ca740 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:25:39.495 7fa7b935b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:25:39.495 7fa7b935b9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:25:39.502 7fa7be3ca740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:25:48.162 7f7d7ebce600 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:25:48.162 7f7d7ebce600 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:25:48.162 7f7d7ebce600 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:25:48.166 7f7d7ebce600 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:25:48.166 7f7d7a1442c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:25:48.166 7f7d7a1442c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:25:48.166 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:25:48.167 7f7d7a1447c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:25:48.167 7f7d7a1447c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:25:48.167 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:25:48.168 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.168 7f7d7a1447c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:25:48.168 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.168 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.168 7f7d7a1447c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352748167700 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:25:48.168 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.169 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.169 7f7d7a1447c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:25:48.169 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.169 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.169 7f7d7a1447c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:48.170 7f7d7a1447c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.170 7f7d7a144e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:25:48.170 7f7d7a144e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:25:48.171 7f7d7a144c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:25:48.172 7f7d7a0f3540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.173 7f7d7a0f3540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:25:48.173 7f7d7a0f3540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.173 7f7d7a0f3540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.173 7f7d7a0f3540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:48.173 7f7d7a0f3540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.226 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:25:48.234 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:25:48.235 7f7d7a094040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.235 7f7d7a094040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:48.235 7f7d7a094040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.235 7f7d7a094040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.235 7f7d7a094040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:48.235 7f7d7a094040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.235 7f7d7a0f3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:48.237 7f7d7a0f3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:48.238 7f7d7a0f37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.238 7f7d7a0f37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:48.238 7f7d7a0f37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.239 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:25:48.240 7f7d7a0f37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.241 7f7d7a0f37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:48.241 7f7d7a0f37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.252 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:25:48.252 7f7d7a094ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.253 7f7d7a094ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:25:48.253 7f7d7a094ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.257 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:25:48.257 7f7d7a094ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.257 7f7d7a094ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:25:48.257 7f7d7a094ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.257 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:25:48.258 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:25:48.258 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:25:48.258 7f7d7a094e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:25:48.258 7f7d79fc70e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.259 7f7d79fc70e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:25:48.259 7f7d79fc70e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.259 7f7d79fc70e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.259 7f7d79fc70e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:25:48.259 7f7d79fc70e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.263 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:25:48.270 7f7d7a094e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:25:48.270 7f7d79fc7d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.271 7f7d79fc7d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:25:48.271 7f7d79fc7d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.289 7f7d7a1447c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:25:48.291 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.291 7f7d79fc7f40 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:25:48.291 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:25:48.359 7f7d7a0f3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:48.359 7f7d7a0f3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:48.689 7f7d7a0f3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:48.690 7f7d7a0f3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:49.172 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:49.177 7f7d7a0f3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:49.180 7f7d7a0f3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:49.509 7f7d79fc7f40 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:25:50.028 7f7d7a0f3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:50.028 7f7d7a0f3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:50.173 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:51.175 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:52.167 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:25:52.169 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:25:52.169 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:25:52.169 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:25:52.169 7f7d79fc7f40 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:25:52.169 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:25:52.169 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:25:52.170 7f7d79fc7f40 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:25:52.170 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:25:52.170 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:25:52.170 7f7d79fc7f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:25:52.170 7f7d79fc7720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:25:52.171 7f7d79fc7720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:25:52.176 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:52.351 7f7d7a0f3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:52.351 7f7d7a0f3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:53.177 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:54.178 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:55.180 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:56.181 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:57.183 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:58.110 7f7d7a0f3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:25:58.111 7f7d7a0f3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:25:58.184 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:25:59.186 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:00.187 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:01.188 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:02.189 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:03.190 7f7d7a144f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:03.233 7f7d7a0f3a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:26:03.234 7f7d7a0f3a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:26:03.234 7f7d7a0f3a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:26:03.342 7f7d7a1447c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:26:03.343 7f7d7ebce600 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:26:03.343 7f7d7ebce600 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:26:03.343 7f7d7a0f39a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:26:03.343 7f7d7a0f39a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:26:03.350 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:26:03.350 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:26:03.350 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:26:03.350 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:26:03.351 7f7d7ebce600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:26:11.888 7f7763c3af80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:26:11.888 7f7763c3af80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:26:11.888 7f7763c3af80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:26:11.892 7f7763c3af80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:26:11.893 7f775f1bc2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:26:11.893 7f775f1bc2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:26:11.893 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:26:11.893 7f775f1bc7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:26:11.894 7f775f1bc7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:26:11.894 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:26:11.894 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.894 7f775f1bc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:26:11.894 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.895 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.895 7f775f1bc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352771894311 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:26:11.895 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.895 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.895 7f775f1bc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:26:11.896 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.896 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.896 7f775f1bc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:11.896 7f775f1bc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.896 7f775f1bce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:26:11.897 7f775f1bce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:26:11.898 7f775f1bcc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:26:11.898 7f775f16c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.899 7f775f16c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:26:11.899 7f775f16c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.899 7f775f16c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.899 7f775f16c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:11.899 7f775f16c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.957 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:26:11.964 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:26:11.965 7f775f10f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.966 7f775f10f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:11.966 7f775f10f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.966 7f775f10f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.966 7f775f10f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:11.966 7f775f10f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.966 7f775f16cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:11.967 7f775f16cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:11.968 7f775f16c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.969 7f775f16c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:11.969 7f775f16c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.970 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:26:11.970 7f775f16c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.970 7f775f16c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:11.970 7f775f16c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.980 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:26:11.981 7f775f10fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.981 7f775f10fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:11.981 7f775f10fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.992 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:26:11.993 7f775f10fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.993 7f775f10fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:11.993 7f775f10fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.994 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:26:11.994 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:26:11.994 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:26:11.994 7f775f10ff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:26:11.995 7f775f041220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.995 7f775f041220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:26:11.995 7f775f041220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.995 7f775f041220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.996 7f775f041220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:26:11.996 7f775f041220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:11.998 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:26:12.002 7f775f10ff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:26:12.003 7f775f041860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:12.003 7f775f041860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:26:12.003 7f775f041860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:12.019 7f775f1bc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:26:12.020 7f775f041e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:12.020 7f775f041e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:26:12.020 7f775f041e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:12.097 7f775f16cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:12.097 7f775f16cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:12.451 7f775f16cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:12.452 7f775f16cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:12.898 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:13.018 7f775f16cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:13.018 7f775f16cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:13.900 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:14.013 7f775f16cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:14.014 7f775f16cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:14.900 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:15.894 7f775f041e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:26:15.895 7f775f041e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:26:15.895 7f775f041e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:26:15.896 7f775f041e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:26:15.896 7f775f041e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:26:15.896 7f775f041e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:26:15.896 7f775f041e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:26:15.896 7f775f041e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:26:15.897 7f775f041e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:26:15.898 7f775f041e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:26:15.898 7f775f041e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:26:15.898 7f7761893360 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:26:15.899 7f7761893360 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:26:15.900 7f775f16cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:15.900 7f775f16cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:15.901 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:16.746 7f775f041e00 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:26:16.903 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:17.904 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:18.905 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:19.906 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:20.176 7f775f16cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:20.176 7f775f16cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:20.907 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:21.908 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:22.909 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:23.910 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:24.911 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:25.912 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:26.905 7f775f16cc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:26:26.906 7f775f16cc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:26:26.906 7f775f16cc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:26:26.914 7f775f1bcf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:27.111 7f775f1bc7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:26:27.111 7f7763c3af80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:26:27.112 7f7763c3af80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:26:27.112 7f775f16cb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:26:27.112 7f775f16cb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:26:27.120 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:26:27.121 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:26:27.122 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:26:27.122 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:26:27.122 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:26:27.122 7f7763c3af80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:26:35.510 7f6366571e80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:26:35.510 7f6366571e80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:26:35.510 7f6366571e80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:26:35.513 7f6366571e80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:26:35.514 7f636197e2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:26:35.514 7f636197e2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:26:35.514 7f636197e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:26:35.514 7f636197e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:26:35.515 7f636197e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:26:35.515 7f636197e7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:26:35.515 7f636197e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.515 7f636197e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:26:35.515 7f636197e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.515 7f636197e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.515 7f636197e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352795514994 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:26:35.515 7f636197e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.516 7f636197e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.516 7f636197e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:26:35.516 7f636197e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.516 7f636197e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.516 7f636197e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:35.516 7f636197e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.516 7f636197ee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:26:35.517 7f636197ee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:26:35.517 7f636197ec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:26:35.518 7f636192d540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.518 7f636192d540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:26:35.518 7f636192d540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.518 7f636192d540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.518 7f636192d540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:35.518 7f636192d540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.572 7f636197e7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:26:35.579 7f636197e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:26:35.580 7f63618cd040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.581 7f63618cd040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:35.581 7f63618cd040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.581 7f63618cd040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.581 7f63618cd040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:35.581 7f63618cd040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.581 7f636192da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:35.582 7f636192da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:35.583 7f636192d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.583 7f636192d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:35.583 7f636192d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.597 7f636197e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:26:35.597 7f636192d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.597 7f636192d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:35.597 7f636192d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.607 7f636197e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:26:35.608 7f63618cdae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.608 7f63618cdae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:35.608 7f63618cdae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.611 7f636197e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:26:35.611 7f63618cdae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.611 7f63618cdae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:35.611 7f63618cdae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.611 7f636197e7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:26:35.612 7f636197e7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:26:35.612 7f636197e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:26:35.612 7f63618cde00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:26:35.613 7f63617fb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.613 7f63617fb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:26:35.613 7f63617fb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.613 7f63617fb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.613 7f63617fb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:26:35.613 7f63617fb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.616 7f636197e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:26:35.619 7f63618cde00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:26:35.619 7f63617fb9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.619 7f63617fb9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:26:35.619 7f63617fb9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.635 7f636197e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:26:35.637 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.637 7f63617fbcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:26:35.637 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:35.723 7f636192da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:35.723 7f636192da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:35.954 7f636192da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:35.955 7f636192da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:36.517 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:36.578 7f636192da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:36.579 7f636192da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:37.469 7f636192da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:37.469 7f636192da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:37.518 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:38.519 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:39.515 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:26:39.516 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:26:39.517 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:26:39.517 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:26:39.517 7f63617fbcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:26:39.517 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:26:39.517 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:26:39.517 7f63617fbcc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:26:39.519 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:26:39.519 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:26:39.519 7f63617fbcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:26:39.519 7f63617fb5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:26:39.520 7f63617fb5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:26:39.520 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:39.545 7f636192da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:39.546 7f636192da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:40.521 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:41.522 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:42.499 7f636192da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:42.499 7f636192da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:42.524 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:43.525 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:44.050 7f63617fb5e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:26:44.526 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:45.527 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:46.528 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:47.530 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:48.531 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:48.846 7f636192da40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:48.847 7f636192da40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:49.532 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:50.533 7f636197ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:26:50.554 7f636192da40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:26:50.554 7f636192da40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:26:50.555 7f636192da40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:26:50.715 7f636197e7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:26:50.715 7f6366571e80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:26:50.716 7f6366571e80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:26:50.716 7f636192d9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:26:50.716 7f636192d9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:26:50.724 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:26:50.725 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:26:50.725 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:26:50.725 7f6366571e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:26:59.186 7fb6442e2a80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:26:59.186 7fb6442e2a80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:26:59.186 7fb6442e2a80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:26:59.189 7fb6442e2a80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:26:59.190 7fb63f8072c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:26:59.190 7fb63f8072c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:26:59.190 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:26:59.190 7fb63f8077c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:26:59.191 7fb63f8077c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:26:59.191 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:26:59.191 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.191 7fb63f8077c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:26:59.191 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.191 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.191 7fb63f8077c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352819190998 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:26:59.191 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.192 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.192 7fb63f8077c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:26:59.192 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.192 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.192 7fb63f8077c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:59.192 7fb63f8077c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.193 7fb63f807e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:26:59.193 7fb63f807e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:26:59.194 7fb63f807c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:26:59.194 7fb63f7b6540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.195 7fb63f7b6540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:26:59.195 7fb63f7b6540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.195 7fb63f7b6540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.195 7fb63f7b6540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:59.195 7fb63f7b6540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.253 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:26:59.261 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:26:59.263 7fb641f520e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.263 7fb641f520e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:59.263 7fb641f520e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.263 7fb641f520e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.263 7fb641f520e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:59.263 7fb641f520e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.263 7fb63f7b6c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:59.265 7fb63f7b6c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:59.266 7fb63f7b67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.267 7fb63f7b67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:59.267 7fb63f7b67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.267 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:26:59.268 7fb63f7b67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.268 7fb63f7b67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:59.268 7fb63f7b67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.279 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:26:59.280 7fb641f52c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.280 7fb641f52c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:26:59.280 7fb641f52c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.283 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:26:59.284 7fb641f52c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.284 7fb641f52c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:26:59.284 7fb641f52c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.284 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:26:59.284 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:26:59.284 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:26:59.285 7fb641f52f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:26:59.285 7fb63f68a220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.285 7fb63f68a220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:26:59.285 7fb63f68a220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.285 7fb63f68a220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.286 7fb63f68a220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:26:59.286 7fb63f68a220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.288 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:26:59.291 7fb641f52f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:26:59.293 7fb63f68aae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.293 7fb63f68aae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:26:59.293 7fb63f68aae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.311 7fb63f8077c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:26:59.314 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.314 7fb63f68ae00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:26:59.314 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:26:59.370 7fb63f7b6c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:59.371 7fb63f7b6c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:26:59.587 7fb63f7b6c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:26:59.587 7fb63f7b6c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:00.032 7fb63f7b6c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:00.033 7fb63f7b6c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:00.193 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:00.678 7fb63f7b6c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:00.678 7fb63f7b6c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:01.195 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:02.130 7fb63f7b6c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:02.131 7fb63f7b6c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:02.196 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:03.191 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:27:03.192 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:27:03.192 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:27:03.192 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:27:03.193 7fb63f68ae00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:27:03.193 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:27:03.193 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:27:03.193 7fb63f68ae00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:27:03.194 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:27:03.195 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:27:03.195 7fb63f68ae00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:27:03.195 7fb63f68a720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:27:03.196 7fb63f68a720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:27:03.197 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:04.199 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:04.685 7fb63f7b6c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:04.686 7fb63f7b6c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:05.200 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:06.201 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:07.203 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:08.204 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:09.206 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:09.817 7fb63f7b6c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:09.817 7fb63f7b6c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:10.207 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:11.255 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:11.838 7fb63ffa9c20 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:27:12.257 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:13.258 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:14.260 7fb63f807f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:14.267 7fb63f7b6c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:27:14.267 7fb63f7b6c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:27:14.268 7fb63f7b6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:27:14.370 7fb63f8077c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:27:14.370 7fb6442e2a80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:27:14.370 7fb6442e2a80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:27:14.370 7fb63f7b6a40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:27:14.370 7fb63f7b6a40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:27:14.377 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:27:14.377 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:27:14.377 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:27:14.378 7fb6442e2a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:27:23.035 7fac77109100 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:27:23.035 7fac77109100 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:27:23.035 7fac77109100 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:27:23.038 7fac77109100 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:27:23.039 7fac71e142c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:27:23.039 7fac71e142c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:27:23.039 7fac71e147c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:27:23.040 7fac71e147c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:27:23.040 7fac71e147c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:27:23.040 7fac71e147c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:27:23.040 7fac71e147c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.040 7fac71e147c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:27:23.040 7fac71e147c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.041 7fac71e147c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.041 7fac71e147c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352843040409 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:27:23.041 7fac71e147c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.041 7fac71e147c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.041 7fac71e147c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:27:23.042 7fac71e147c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.043 7fac71e147c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.043 7fac71e147c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:27:23.043 7fac71e147c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.043 7fac71e14f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:27:23.044 7fac71e14f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:27:23.045 7fac71e14c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:27:23.045 7fac71dc37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.045 7fac71dc37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:27:23.045 7fac71dc37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.045 7fac71dc37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.046 7fac71dc37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:27:23.046 7fac71dc37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.099 7fac71e147c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:27:23.107 7fac71e147c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:27:23.108 7fac71d680e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.108 7fac71d680e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:27:23.108 7fac71d680e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.109 7fac71d680e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.109 7fac71d680e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:27:23.109 7fac71d680e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.109 7fac71dc3c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:23.110 7fac71dc3c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:23.111 7fac71dc39a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.111 7fac71dc39a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:27:23.111 7fac71dc39a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.112 7fac71e147c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:27:23.113 7fac71dc39a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.113 7fac71dc39a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:27:23.113 7fac71dc39a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.123 7fac71e147c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:27:23.123 7fac71d68c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.124 7fac71d68c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:27:23.124 7fac71d68c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.126 7fac71e147c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:27:23.126 7fac71d68c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.126 7fac71d68c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:27:23.126 7fac71d68c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.127 7fac71e147c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:27:23.127 7fac71e147c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:27:23.127 7fac71e147c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:27:23.127 7fac71d68f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:27:23.128 7fac71c98220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.128 7fac71c98220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:27:23.128 7fac71c98220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.128 7fac71c98220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.128 7fac71c98220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:27:23.128 7fac71c98220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.131 7fac71e147c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:27:23.134 7fac71d68f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:27:23.134 7fac71c98ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.134 7fac71c98ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:27:23.134 7fac71c98ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.149 7fac71e147c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:27:23.150 7fac71c98e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.150 7fac71c98e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:27:23.151 7fac71c98e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:23.180 7fac71dc3c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:23.181 7fac71dc3c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:23.269 7fac71dc3c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:23.270 7fac71dc3c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:23.497 7fac71dc3c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:23.498 7fac71dc3c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:24.044 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:24.071 7fac71dc3c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:24.072 7fac71dc3c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:24.847 7fac71dc3c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:24.847 7fac71dc3c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:25.045 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:26.046 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:26.345 7fac71dc3c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:26.346 7fac71dc3c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:27.041 7fac71c98e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:27:27.042 7fac71c98e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:27:27.042 7fac71c98e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:27:27.042 7fac71c98e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:27:27.042 7fac71c98e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:27:27.042 7fac71c98e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:27:27.042 7fac71c98e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:27:27.043 7fac71c98e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:27:27.044 7fac71c98e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:27:27.044 7fac71c98e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:27:27.044 7fac71c98e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:27:27.045 7fac71c98720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:27:27.046 7fac71c98720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:27:27.047 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:28.048 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:29.049 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:29.451 7fac71dc3c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:29.451 7fac71dc3c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:30.050 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:31.051 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:32.052 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:33.053 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:34.054 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:35.055 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:36.056 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:37.057 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:37.463 7fac71dc3c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:37.464 7fac71dc3c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:38.058 7fac71dc32c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:38.066 7fac71dc3c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:27:38.066 7fac71dc3c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:27:38.067 7fac71dc3c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:27:38.232 7fac71e147c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:27:38.232 7fac77109100 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:27:38.232 7fac77109100 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:27:38.232 7fac71dc3b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:27:38.232 7fac71dc3b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:27:38.239 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:27:38.240 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:27:38.241 7fac77109100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:27:46.972 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:27:46.972 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:27:46.972 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:27:46.975 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:27:46.975 7fbaf04de2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:27:46.976 7fbaf04de2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:27:46.976 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:27:46.976 7fbaf04de7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:27:46.976 7fbaf04de7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:27:46.977 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:27:46.977 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.977 7fbaf04de7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:27:46.977 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.977 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.977 7fbaf04de7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352866976887 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:27:46.977 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.978 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.978 7fbaf04de7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:27:46.978 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.978 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.978 7fbaf04de7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:27:46.978 7fbaf04de7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.978 7fbaf04dee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:27:46.979 7fbaf04dee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:27:46.980 7fbaf04dec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:27:46.980 7fbaf048e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.980 7fbaf048e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:27:46.980 7fbaf048e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.980 7fbaf048e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:46.980 7fbaf048e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:27:46.980 7fbaf048e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.036 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:27:47.044 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:27:47.045 7fbaf25b7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.045 7fbaf25b7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:27:47.045 7fbaf25b7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.046 7fbaf25b7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.046 7fbaf25b7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:27:47.046 7fbaf25b7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.046 7fbaf048ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:47.047 7fbaf048ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:47.048 7fbaf048e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.048 7fbaf048e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:27:47.048 7fbaf048e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.049 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:27:47.049 7fbaf048e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.049 7fbaf048e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:27:47.049 7fbaf048e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.059 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:27:47.060 7fbaf25b7ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.060 7fbaf25b7ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:27:47.060 7fbaf25b7ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.062 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:27:47.063 7fbaf25b7ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.063 7fbaf25b7ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:27:47.063 7fbaf25b7ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.063 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:27:47.063 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:27:47.064 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:27:47.064 7fbaf25b7e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:27:47.064 7fbaf1ab30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.064 7fbaf1ab30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:27:47.065 7fbaf1ab30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.065 7fbaf1ab30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.065 7fbaf1ab30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:27:47.065 7fbaf1ab30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.067 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:27:47.070 7fbaf25b7e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:27:47.071 7fbaf1ab39a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.071 7fbaf1ab39a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:27:47.071 7fbaf1ab39a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.086 7fbaf04de7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:27:47.088 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.088 7fbaf1ab3cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:27:47.088 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:27:47.160 7fbaf048ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:47.161 7fbaf048ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:47.469 7fbaf048ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:47.470 7fbaf048ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:47.980 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:48.002 7fbaf048ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:48.003 7fbaf048ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:48.981 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:49.148 7fbaf048ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:49.149 7fbaf048ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:49.983 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:50.977 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:27:50.978 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:27:50.979 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:27:50.979 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:27:50.979 7fbaf1ab3cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:27:50.979 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:27:50.979 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:27:50.979 7fbaf1ab3cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:27:50.981 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:27:50.981 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:27:50.981 7fbaf1ab3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:27:50.981 7fbaf1ab35e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:27:50.982 7fbaf1ab35e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:27:50.984 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:51.232 7fbaf048ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:51.232 7fbaf048ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:51.985 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:52.986 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:53.777 7fbaf048ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:53.778 7fbaf048ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:53.987 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:54.988 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:55.989 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:56.991 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:57.992 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:58.994 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:27:59.708 7fbaf048ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:27:59.708 7fbaf048ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:27:59.995 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:00.996 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:01.997 7fbaf04def40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:02.017 7fbaf048ea40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:28:02.018 7fbaf048ea40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:28:02.019 7fbaf048ea40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:28:02.114 7fbaf04de7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:28:02.114 7fbaf4f655c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:28:02.114 7fbaf4f655c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:28:02.114 7fbaf048e9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:28:02.115 7fbaf048e9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:28:02.122 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:28:02.123 7fbaf4f655c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:28:10.894 7ff51f594f80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:28:10.894 7ff51f594f80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:28:10.894 7ff51f594f80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:28:10.898 7ff51f594f80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:28:10.898 7ff51ab0f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:28:10.898 7ff51ab0f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:28:10.898 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:28:10.899 7ff51ab0f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:28:10.899 7ff51ab0f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:28:10.899 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:28:10.899 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.900 7ff51ab0f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:28:10.900 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.900 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.900 7ff51ab0f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352890899560 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:28:10.900 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.900 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.900 7ff51ab0f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:28:10.900 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.901 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.901 7ff51ab0f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:10.901 7ff51ab0f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.901 7ff51ab0fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:28:10.901 7ff51ab0fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:28:10.902 7ff51ab0fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:28:10.903 7ff51aabe540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.903 7ff51aabe540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:28:10.903 7ff51aabe540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.903 7ff51aabe540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.903 7ff51aabe540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:10.904 7ff51aabe540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.958 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:28:10.966 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:28:10.967 7ff51aaa30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.968 7ff51aaa30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:10.968 7ff51aaa30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.968 7ff51aaa30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.968 7ff51aaa30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:10.968 7ff51aaa30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.968 7ff51aabec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:10.969 7ff51aabec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:10.970 7ff51aabe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.970 7ff51aabe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:10.970 7ff51aabe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.971 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:28:10.972 7ff51aabe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.972 7ff51aabe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:10.972 7ff51aabe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.982 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:28:10.982 7ff51aaa3c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.983 7ff51aaa3c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:10.983 7ff51aaa3c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.985 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:28:10.986 7ff51aaa3c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.986 7ff51aaa3c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:10.986 7ff51aaa3c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.986 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:28:10.986 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:28:10.986 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:28:10.987 7ff51aaa3f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:28:10.987 7ff51a993220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.987 7ff51a993220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:28:10.987 7ff51a993220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.988 7ff51a993220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.988 7ff51a993220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:28:10.988 7ff51a993220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.990 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:28:10.993 7ff51aaa3f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:28:10.994 7ff51a993ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:10.994 7ff51a993ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:28:10.994 7ff51a993ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:11.011 7ff51ab0f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:28:11.012 7ff51a993e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:11.012 7ff51a993e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:28:11.012 7ff51a993e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:11.103 7ff51aabec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:11.104 7ff51aabec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:11.394 7ff51aabec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:11.394 7ff51aabec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:11.757 7ff51aabec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:11.757 7ff51aabec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:11.902 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:12.626 7ff51aabec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:12.631 7ff51aabec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:12.904 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:13.905 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:14.810 7ff51aabec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:14.811 7ff51aabec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:14.900 7ff51a993e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:28:14.901 7ff51a993e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:28:14.901 7ff51a993e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:28:14.901 7ff51a993e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:28:14.902 7ff51a993e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:28:14.902 7ff51a993e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:28:14.902 7ff51a993e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:28:14.902 7ff51a993e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:28:14.904 7ff51a993e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:28:14.904 7ff51a993e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:28:14.904 7ff51a993e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:28:14.904 7ff51a993720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:28:14.905 7ff51a993720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:28:14.906 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:15.908 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:16.909 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:17.910 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:18.605 7ff51aabec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:18.606 7ff51aabec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:18.912 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:19.913 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:20.914 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:21.914 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:22.916 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:23.917 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:24.919 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:25.920 7ff51ab0ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:25.938 7ff51aabec20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:28:25.939 7ff51aabec20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:28:25.939 7ff51b2b1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:28:26.065 7ff51ab0f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:28:26.065 7ff51f594f80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:28:26.065 7ff51f594f80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:28:26.065 7ff51aabeb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:28:26.065 7ff51aabeb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:28:26.072 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:28:26.073 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:28:26.074 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:28:26.074 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:28:26.074 7ff51f594f80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:28:34.624 7fa4b3352f40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:28:34.625 7fa4b3352f40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:28:34.625 7fa4b3352f40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:28:34.628 7fa4b3352f40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:28:34.629 7fa4ae8b92c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:28:34.629 7fa4ae8b92c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:28:34.629 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:28:34.630 7fa4ae8b97c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:28:34.630 7fa4ae8b97c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:28:34.630 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:28:34.630 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.630 7fa4ae8b97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:28:34.631 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.631 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.631 7fa4ae8b97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352914630399 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:28:34.631 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.631 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.631 7fa4ae8b97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:28:34.631 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.632 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.632 7fa4ae8b97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:34.632 7fa4ae8b97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.632 7fa4ae8b9e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:28:34.632 7fa4ae8b9e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:28:34.633 7fa4ae8b9c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:28:34.634 7fa4ae869540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.634 7fa4ae869540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:28:34.634 7fa4ae869540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.634 7fa4ae869540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.634 7fa4ae869540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:34.634 7fa4ae869540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.693 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:28:34.700 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:28:34.701 7fa4b09cf040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.701 7fa4b09cf040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:34.701 7fa4b09cf040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.701 7fa4b09cf040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.702 7fa4b09cf040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:34.702 7fa4b09cf040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.702 7fa4ae869a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:34.703 7fa4ae869a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:34.704 7fa4ae8697c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.704 7fa4ae8697c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:34.704 7fa4ae8697c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.705 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:28:34.705 7fa4ae8697c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.705 7fa4ae8697c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:34.705 7fa4ae8697c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.715 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:28:34.716 7fa4b09cfae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.716 7fa4b09cfae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:34.716 7fa4b09cfae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.719 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:28:34.719 7fa4b09cfae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.719 7fa4b09cfae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:34.719 7fa4b09cfae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.720 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:28:34.720 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:28:34.720 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:28:34.720 7fa4b09cfe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:28:34.721 7fa4afe930e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.721 7fa4afe930e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:28:34.721 7fa4afe930e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.721 7fa4afe930e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.721 7fa4afe930e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:28:34.721 7fa4afe930e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.724 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:28:34.727 7fa4b09cfe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:28:34.727 7fa4afe939a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.727 7fa4afe939a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:28:34.727 7fa4afe939a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.742 7fa4ae8b97c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:28:34.744 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.744 7fa4afe93cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:28:34.744 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:34.767 7fa4ae869a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:34.768 7fa4ae869a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:34.924 7fa4ae869a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:34.925 7fa4ae869a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:35.285 7fa4ae869a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:35.286 7fa4ae869a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:35.633 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:36.043 7fa4ae869a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:36.044 7fa4ae869a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:36.635 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:36.989 7fa4ae869a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:36.990 7fa4ae869a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:37.635 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:38.630 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:28:38.631 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:28:38.632 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:28:38.632 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:28:38.632 7fa4afe93cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:28:38.632 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:28:38.632 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:28:38.632 7fa4afe93cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:28:38.634 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:28:38.634 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:28:38.634 7fa4afe93cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:28:38.634 7fa4afe935e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:28:38.635 7fa4afe935e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:28:38.637 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:39.440 7fa4ae869a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:39.440 7fa4ae869a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:39.638 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:40.640 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:41.641 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:42.642 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:43.644 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:44.645 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:44.822 7fa4ae869a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:44.823 7fa4ae869a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:45.646 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:46.648 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:47.649 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:48.650 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:49.645 7fa4ae869a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:28:49.645 7fa4ae869a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:28:49.646 7fa4ae869a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:28:49.651 7fa4ae8b9f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:28:49.761 7fa4ae8b97c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:28:49.761 7fa4b3352f40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:28:49.761 7fa4b3352f40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:28:49.761 7fa4ae8699a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:28:49.761 7fa4ae8699a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:28:49.768 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:28:49.769 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:28:49.769 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:28:49.769 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:28:49.769 7fa4b3352f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:28:58.420 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:28:58.420 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:28:58.420 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:28:58.423 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:28:58.423 7f7b357252c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:28:58.423 7f7b357252c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:28:58.424 7f7b357257c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:28:58.424 7f7b357257c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:28:58.424 7f7b357257c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:28:58.424 7f7b357257c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:28:58.425 7f7b357257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.425 7f7b357257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:28:58.425 7f7b357257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.425 7f7b357257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.425 7f7b357257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352938424689 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:28:58.425 7f7b357257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.425 7f7b357257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.426 7f7b357257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:28:58.426 7f7b357257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.426 7f7b357257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.426 7f7b357257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:58.426 7f7b357257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.426 7f7b35725e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:28:58.426 7f7b35725e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:28:58.428 7f7b35725c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:28:58.428 7f7b356d4540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.428 7f7b356d4540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:28:58.428 7f7b356d4540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.429 7f7b356d4540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.430 7f7b356d4540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:58.430 7f7b356d4540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.482 7f7b357257c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:28:58.489 7f7b357257c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:28:58.490 7f7b356790e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.490 7f7b356790e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:58.490 7f7b356790e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.491 7f7b356790e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.491 7f7b356790e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:58.491 7f7b356790e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.491 7f7b356d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:58.492 7f7b356d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:58.493 7f7b356d47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.493 7f7b356d47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:58.493 7f7b356d47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.494 7f7b357257c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:28:58.494 7f7b356d47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.494 7f7b356d47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:58.494 7f7b356d47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.503 7f7b357257c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:28:58.503 7f7b35679c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.504 7f7b35679c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:28:58.504 7f7b35679c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.506 7f7b357257c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:28:58.506 7f7b35679c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.506 7f7b35679c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:28:58.506 7f7b35679c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.507 7f7b357257c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:28:58.507 7f7b357257c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:28:58.507 7f7b357257c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:28:58.507 7f7b35679f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:28:58.508 7f7b355a8220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.508 7f7b355a8220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:28:58.508 7f7b355a8220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.508 7f7b355a8220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.508 7f7b355a8220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:28:58.508 7f7b355a8220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.510 7f7b357257c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:28:58.514 7f7b35679f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:28:58.515 7f7b355a8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.515 7f7b355a8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:28:58.515 7f7b355a8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.530 7f7b357257c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:28:58.534 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.534 7f7b355a8cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:28:58.534 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:28:58.597 7f7b356d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:58.597 7f7b356d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:58.876 7f7b356d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:58.877 7f7b356d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:59.327 7f7b356d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:28:59.328 7f7b356d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:28:59.428 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:00.041 7f7b356d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:00.042 7f7b356d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:00.429 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:01.283 7f7b355a8cc0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:29:01.431 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:01.535 7f7b356d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:01.536 7f7b356d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:02.424 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:29:02.425 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:29:02.426 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:29:02.426 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:02.426 7f7b355a8cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:29:02.426 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:02.426 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:29:02.426 7f7b355a8cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:29:02.426 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:29:02.426 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:29:02.426 7f7b355a8cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:29:02.427 7f7b355a8860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:29:02.427 7f7b355a8860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:29:02.432 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:03.433 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:03.653 7f7b356d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:03.653 7f7b356d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:04.434 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:05.435 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:06.437 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:07.439 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:07.790 7f7b356d4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:07.791 7f7b356d4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:08.440 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:09.441 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:10.442 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:11.443 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:12.444 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:13.446 7f7b35725f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:13.515 7f7b356d4c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:29:13.516 7f7b356d4c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:29:13.516 7f7b35ec7c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:29:13.564 7f7b357257c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:29:13.564 7f7b3a1a9a40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:29:13.565 7f7b3a1a9a40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:29:13.565 7f7b356d4680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:29:13.565 7f7b356d4680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:29:13.572 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:29:13.573 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:29:13.573 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:29:13.573 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:29:13.573 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:29:13.573 7f7b3a1a9a40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:29:22.130 7fdd67ba9800 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:29:22.130 7fdd67ba9800 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:29:22.130 7fdd67ba9800 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:29:22.133 7fdd67ba9800 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:29:22.134 7fdd62f452c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:29:22.134 7fdd62f452c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:29:22.134 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:29:22.135 7fdd62f457c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:29:22.135 7fdd62f457c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:29:22.135 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:29:22.135 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.135 7fdd62f457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:29:22.135 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.135 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.135 7fdd62f457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352962135312 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:29:22.136 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.136 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.136 7fdd62f457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:29:22.136 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.136 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.136 7fdd62f457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:29:22.136 7fdd62f457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.137 7fdd62f45e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:29:22.137 7fdd62f45e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:29:22.138 7fdd62f45c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:29:22.138 7fdd62ef5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.138 7fdd62ef5540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:29:22.139 7fdd62ef5540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.139 7fdd62ef5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.139 7fdd62ef5540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:29:22.139 7fdd62ef5540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.194 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:29:22.202 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:29:22.203 7fdd64527040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.203 7fdd64527040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:29:22.203 7fdd64527040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.203 7fdd64527040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.203 7fdd64527040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:29:22.203 7fdd64527040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.204 7fdd62ef5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:22.205 7fdd62ef5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:22.205 7fdd62ef57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.205 7fdd62ef57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:29:22.206 7fdd62ef57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.207 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:29:22.207 7fdd62ef57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.207 7fdd62ef57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:29:22.207 7fdd62ef57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.216 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:29:22.217 7fdd64527ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.217 7fdd64527ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:29:22.217 7fdd64527ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.220 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:29:22.220 7fdd64527ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.220 7fdd64527ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:29:22.220 7fdd64527ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.221 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:29:22.221 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:29:22.221 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:29:22.221 7fdd64527e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:29:22.222 7fdd645bc0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.222 7fdd645bc0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:29:22.222 7fdd645bc0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.222 7fdd645bc0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.222 7fdd645bc0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:29:22.222 7fdd645bc0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.225 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:29:22.235 7fdd64527e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:29:22.235 7fdd64527cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.236 7fdd64527cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:29:22.236 7fdd64527cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.245 7fdd62f457c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:29:22.246 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.247 7fdd645bccc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:29:22.247 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:22.316 7fdd62ef5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:22.317 7fdd62ef5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:22.552 7fdd62ef5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:22.553 7fdd62ef5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:22.949 7fdd62ef5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:22.950 7fdd62ef5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:23.138 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:23.946 7fdd62ef5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:23.946 7fdd62ef5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:24.139 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:25.140 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:26.135 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:29:26.137 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:29:26.137 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:29:26.137 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:26.137 7fdd645bccc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:29:26.137 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:26.137 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:29:26.137 7fdd645bccc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:29:26.139 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:29:26.139 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:29:26.139 7fdd645bccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:29:26.139 7fdd645bc680 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:29:26.140 7fdd645bc680 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:29:26.141 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:26.439 7fdd62ef5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:26.440 7fdd62ef5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:27.141 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:28.142 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:28.547 7fdd645bc680 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:29:29.143 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:29.477 7fdd62ef5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:29.478 7fdd62ef5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:30.145 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:31.146 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:32.147 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:33.149 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:34.149 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:35.151 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:36.152 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:36.352 7fdd62ef5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:36.352 7fdd62ef5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:37.153 7fdd62f45f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:37.155 7fdd62ef5a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:29:37.156 7fdd62ef5a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:29:37.156 7fdd62ef5a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:29:37.343 7fdd62f457c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:29:37.343 7fdd67ba9800 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:29:37.343 7fdd67ba9800 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:29:37.344 7fdd62ef59a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:29:37.344 7fdd62ef59a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:29:37.350 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:29:37.350 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:29:37.350 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:29:37.350 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:29:37.350 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:29:37.350 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:29:37.350 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:29:37.350 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:29:37.350 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:29:37.351 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:29:37.351 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:29:37.351 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:29:37.351 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:29:37.351 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:29:37.351 7fdd67ba9800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:29:45.576 7f4648697e80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:29:45.576 7f4648697e80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:29:45.576 7f4648697e80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:29:45.579 7f4648697e80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:29:45.580 7f4643c112c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:29:45.580 7f4643c112c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:29:45.580 7f4643c117c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:29:45.580 7f4643c117c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:29:45.581 7f4643c117c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:29:45.581 7f4643c117c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:29:45.581 7f4643c117c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.581 7f4643c117c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:29:45.581 7f4643c117c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.581 7f4643c117c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.581 7f4643c117c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789352985580975 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:29:45.581 7f4643c117c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.581 7f4643c117c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.582 7f4643c117c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:29:45.582 7f4643c117c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.582 7f4643c117c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.582 7f4643c117c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:29:45.582 7f4643c117c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.582 7f4643c11e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:29:45.582 7f4643c11e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:29:45.583 7f4643c11c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:29:45.584 7f4643bc1540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.584 7f4643bc1540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:29:45.584 7f4643bc1540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.584 7f4643bc1540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.584 7f4643bc1540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:29:45.584 7f4643bc1540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.637 7f4643c117c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:29:45.644 7f4643c117c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:29:45.645 7f4643b65040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.645 7f4643b65040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:29:45.645 7f4643b65040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.645 7f4643b65040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.646 7f4643b65040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:29:45.646 7f4643b65040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.646 7f4643bc1a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:45.647 7f4643bc1a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:45.648 7f4643bc17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.648 7f4643bc17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:29:45.648 7f4643bc17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.649 7f4643c117c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:29:45.649 7f4643bc17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.649 7f4643bc17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:29:45.649 7f4643bc17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.658 7f4643c117c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:29:45.659 7f4643b65ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.659 7f4643b65ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:29:45.659 7f4643b65ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.661 7f4643c117c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:29:45.661 7f4643b65ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.662 7f4643b65ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:29:45.662 7f4643b65ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.662 7f4643c117c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:29:45.662 7f4643c117c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:29:45.662 7f4643c117c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:29:45.662 7f4643b65e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:29:45.663 7f4643a950e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.663 7f4643a950e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:29:45.663 7f4643a950e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.663 7f4643a950e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.663 7f4643a950e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:29:45.663 7f4643a950e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.665 7f4643c117c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:29:45.668 7f4643b65e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:29:45.668 7f4643a959a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.668 7f4643a959a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:29:45.669 7f4643a959a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.684 7f4643c117c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:29:45.685 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.685 7f4643a95cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:29:45.685 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:45.762 7f4643bc1a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:45.763 7f4643bc1a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:46.076 7f4643bc1a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:46.077 7f4643bc1a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:46.583 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:46.873 7f4643bc1a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:46.873 7f4643bc1a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:47.584 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:48.586 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:48.838 7f4643bc1a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:48.839 7f4643bc1a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:49.581 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:29:49.582 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:29:49.582 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:29:49.582 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:49.582 7f4643a95cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:29:49.582 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:49.583 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:29:49.583 7f4643a95cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:29:49.584 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:29:49.584 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:29:49.584 7f4643a95cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:29:49.585 7f4643a955e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:29:49.586 7f4643a955e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:29:49.586 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:50.587 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:51.588 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:52.173 7f4643bc1a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:52.174 7f4643bc1a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:52.590 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:53.591 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:54.591 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:55.593 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:56.208 7f4643a955e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:29:56.594 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:57.595 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:58.597 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:29:58.895 7f4643bc1a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:29:58.896 7f4643bc1a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:29:59.597 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:00.598 7f4643c11f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:00.603 7f4643bc1a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:30:00.603 7f4643bc1a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:30:00.604 7f4643bc1a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:00.771 7f4643c117c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:30:00.772 7f4648697e80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:30:00.772 7f4648697e80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:30:00.772 7f4643bc19a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:30:00.772 7f4643bc19a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:30:00.779 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:30:00.779 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:30:00.779 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:30:00.779 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:30:00.779 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:30:00.780 7f4648697e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:30:09.304 7f8d2f176a80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:30:09.304 7f8d2f176a80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:30:09.304 7f8d2f176a80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:30:09.308 7f8d2f176a80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:30:09.308 7f8d2a5d42c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:30:09.308 7f8d2a5d42c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:30:09.309 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:30:09.309 7f8d2a5d47c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:30:09.309 7f8d2a5d47c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:30:09.310 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:30:09.310 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.310 7f8d2a5d47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:30:09.310 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.310 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.310 7f8d2a5d47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353009309769 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:30:09.310 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.311 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.311 7f8d2a5d47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:30:09.311 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.311 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.311 7f8d2a5d47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:09.311 7f8d2a5d47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.312 7f8d2a5d4e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:09.312 7f8d2a5d4e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:30:09.313 7f8d2a5d4c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:30:09.314 7f8d2a583540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.314 7f8d2a583540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:09.314 7f8d2a583540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.314 7f8d2a583540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.314 7f8d2a583540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:09.314 7f8d2a583540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.375 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:30:09.384 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:30:09.385 7f8d2a524040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.385 7f8d2a524040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:09.385 7f8d2a524040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.385 7f8d2a524040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.386 7f8d2a524040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:09.386 7f8d2a524040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.386 7f8d2a583a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:09.387 7f8d2a583a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:09.388 7f8d2a5837c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.388 7f8d2a5837c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:09.388 7f8d2a5837c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.389 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:30:09.390 7f8d2a5837c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.390 7f8d2a5837c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:09.390 7f8d2a5837c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.400 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:09.400 7f8d2a524ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.401 7f8d2a524ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:09.401 7f8d2a524ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.403 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:09.403 7f8d2a524ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.403 7f8d2a524ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:09.403 7f8d2a524ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.404 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:30:09.404 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:30:09.404 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:30:09.404 7f8d2a524e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:09.405 7f8d2a4510e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.405 7f8d2a4510e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:09.405 7f8d2a4510e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.405 7f8d2a4510e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.405 7f8d2a4510e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:09.405 7f8d2a4510e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.408 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:30:09.410 7f8d2a524e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:09.411 7f8d2a4519a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.411 7f8d2a4519a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:09.411 7f8d2a4519a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.425 7f8d2a5d47c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:30:09.427 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.427 7f8d2a451cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:30:09.427 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.490 7f8d2a583a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:09.491 7f8d2a583a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:09.693 7f8d2a583a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:09.694 7f8d2a583a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:10.031 7f8d2a583a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:10.032 7f8d2a583a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:10.313 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:10.836 7f8d2a583a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:10.837 7f8d2a583a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:11.314 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:12.315 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:12.944 7f8d2a583a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:12.945 7f8d2a583a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:13.310 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:13.311 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:30:13.311 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:13.312 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:13.312 7f8d2a451cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:30:13.312 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:13.312 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:30:13.312 7f8d2a451cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:30:13.314 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:30:13.314 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:13.314 7f8d2a451cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:13.314 7f8d2c6ce220 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:30:13.315 7f8d2c6ce220 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:30:13.317 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:14.318 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:15.319 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:16.321 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:17.322 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:17.699 7f8d2a583a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:17.700 7f8d2a583a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:18.323 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:19.324 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:20.325 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:21.326 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:22.328 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:23.329 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:23.458 7f8d2a451cc0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:30:24.329 7f8d2a583a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:30:24.329 7f8d2a583a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:30:24.330 7f8d2a583a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:24.331 7f8d2a5d4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:24.529 7f8d2a5d47c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:30:24.529 7f8d2f176a80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:30:24.530 7f8d2f176a80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:30:24.530 7f8d2a5839a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:30:24.530 7f8d2a5839a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:30:24.538 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:30:24.538 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:30:24.538 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:30:24.538 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:30:24.538 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:30:24.538 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:30:24.538 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:30:24.538 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:30:24.538 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:30:24.539 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:30:24.539 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:30:24.539 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:30:24.539 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:30:24.539 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:30:24.539 7f8d2f176a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:30:32.923 7f67d972ee80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:30:32.923 7f67d972ee80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:30:32.923 7f67d972ee80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:30:32.926 7f67d972ee80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:30:32.926 7f67d4bbb2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:30:32.926 7f67d4bbb2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:30:32.927 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:30:32.927 7f67d4bbb7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:30:32.928 7f67d4bbb7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:30:32.928 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:30:32.928 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.928 7f67d4bbb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:30:32.928 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.928 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.928 7f67d4bbb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353032928149 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:30:32.929 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.929 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.929 7f67d4bbb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:30:32.929 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.929 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.930 7f67d4bbb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:32.930 7f67d4bbb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.930 7f67d4bbbf40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:32.930 7f67d4bbbf40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:30:32.931 7f67d4bbbe00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:30:32.932 7f67d4b6b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.932 7f67d4b6b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:32.932 7f67d4b6b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.932 7f67d4b6b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.932 7f67d4b6b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:32.932 7f67d4b6b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.989 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:30:32.997 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:30:32.998 7f67d4b0a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.998 7f67d4b0a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:32.998 7f67d4b0a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.998 7f67d4b0a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.998 7f67d4b0a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:32.998 7f67d4b0a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:32.999 7f67d4b6bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:33.000 7f67d4b0a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.000 7f67d4b0a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:33.000 7f67d4b0a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.000 7f67d4b6bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:33.002 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:30:33.004 7f67d4b6b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.004 7f67d4b6b9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:33.004 7f67d4b6b9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.013 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:33.014 7f67d4b0ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.014 7f67d4b0ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:33.014 7f67d4b0ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.017 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:33.017 7f67d4b0ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.017 7f67d4b0ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:33.017 7f67d4b0ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.017 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:30:33.018 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:30:33.018 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:30:33.018 7f67d4b0af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:33.019 7f67d4a3f220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.019 7f67d4a3f220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:33.019 7f67d4a3f220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.019 7f67d4a3f220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.019 7f67d4a3f220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:33.019 7f67d4a3f220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.022 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:30:33.027 7f67d4b0af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:33.027 7f67d4a3fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.028 7f67d4a3fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:33.028 7f67d4a3fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.043 7f67d4bbb7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:30:33.045 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.046 7f67d4a3fe00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:30:33.046 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:33.062 7f67d4b6bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:33.062 7f67d4b6bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:33.220 7f67d4b6bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:33.221 7f67d4b6bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:33.643 7f67d4b6bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:33.644 7f67d4b6bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:33.931 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:34.292 7f67d4b6bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:34.293 7f67d4b6bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:34.933 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:35.816 7f67d4b6bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:35.817 7f67d4b6bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:35.934 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:36.928 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:36.929 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:30:36.929 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:36.930 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:36.930 7f67d4a3fe00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:30:36.930 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:36.930 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:30:36.930 7f67d4a3fe00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:30:36.932 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:30:36.932 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:36.932 7f67d4a3fe00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:36.932 7f67d4a3f720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:30:36.933 7f67d4a3f720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:30:36.935 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:37.936 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:38.937 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:39.087 7f67d4b6bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:39.087 7f67d4b6bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:39.938 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:40.939 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:41.940 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:42.941 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:43.937 7f67d4b6bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:43.938 7f67d4b6bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:43.942 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:44.943 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:45.944 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:46.945 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:47.946 7f67d4b6b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:47.954 7f67d4b6bc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:30:47.954 7f67d4b6bc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:30:47.955 7f67d4b6bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:48.117 7f67d4bbb7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:30:48.117 7f67d972ee80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:30:48.118 7f67d972ee80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:30:48.118 7f67d4b6bb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:30:48.118 7f67d4b6bb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:30:48.125 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:30:48.125 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:30:48.125 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:30:48.125 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:30:48.125 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:30:48.125 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:30:48.125 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:30:48.126 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:30:48.126 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:30:48.126 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:30:48.126 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:30:48.126 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:30:48.126 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:30:48.126 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:30:48.126 7f67d972ee80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:30:56.467 7f8b72ccc180 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:30:56.467 7f8b72ccc180 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:30:56.467 7f8b72ccc180 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:30:56.470 7f8b72ccc180 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:30:56.470 7f8b6e2462c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:30:56.470 7f8b6e2462c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:30:56.471 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:30:56.471 7f8b6e2467c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:30:56.471 7f8b6e2467c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:30:56.472 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:30:56.472 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.472 7f8b6e2467c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:30:56.472 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.472 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.472 7f8b6e2467c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353056471757 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:30:56.472 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.472 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.473 7f8b6e2467c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:30:56.473 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.473 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.473 7f8b6e2467c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:56.473 7f8b6e2467c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.473 7f8b6e246e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:56.473 7f8b6e246e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:30:56.474 7f8b6e246c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:30:56.474 7f8b6e1f5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.475 7f8b6e1f5540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:56.475 7f8b6e1f5540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.475 7f8b6e1f5540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.475 7f8b6e1f5540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:56.475 7f8b6e1f5540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.527 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:30:56.534 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:30:56.535 7f8b6e194040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.535 7f8b6e194040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:56.535 7f8b6e194040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.535 7f8b6e194040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.535 7f8b6e194040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:56.535 7f8b6e194040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.536 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:56.537 7f8b6e1f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.537 7f8b6e1f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:56.537 7f8b6e1f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.537 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:56.538 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:30:56.539 7f8b6e1f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.539 7f8b6e1f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:56.539 7f8b6e1f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.548 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:56.549 7f8b6e194ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.549 7f8b6e194ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:56.549 7f8b6e194ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.551 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:56.551 7f8b6e194ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.551 7f8b6e194ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:56.551 7f8b6e194ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.552 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:30:56.552 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:30:56.552 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:30:56.552 7f8b6e194e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:56.553 7f8b6e0c30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.553 7f8b6e0c30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:56.553 7f8b6e0c30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.553 7f8b6e0c30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.553 7f8b6e0c30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:56.553 7f8b6e0c30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.555 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:30:56.558 7f8b6e194e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:56.559 7f8b6e0c3720 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.559 7f8b6e0c3720 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:56.559 7f8b6e0c3720 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.574 7f8b6e2467c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:30:56.575 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.575 7f8b6e0c3cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:30:56.575 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:56.623 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:56.624 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:56.802 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:56.803 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:57.024 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:57.025 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:57.304 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:57.305 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:57.474 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:57.666 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:57.667 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:58.174 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:58.175 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:58.476 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:59.338 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:30:59.339 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:59.477 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:00.472 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:00.473 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:00.473 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:00.473 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:00.474 7f8b6e0c3cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:00.474 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:00.474 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:00.474 7f8b6e0c3cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:31:00.476 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:00.476 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:00.476 7f8b6e0c3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:00.476 7f8b6e0c35e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:31:00.477 7f8b6e0c35e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:31:00.479 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:00.822 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:00.823 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:01.480 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:02.481 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:02.816 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:02.817 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:03.482 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:04.483 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:05.484 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:06.485 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:07.361 7f8b6e1f5a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:07.362 7f8b6e1f5a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.485 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:08.487 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:09.488 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:10.490 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:11.479 7f8b6e1f5a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:31:11.479 7f8b6e1f5a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:31:11.480 7f8b6e1f5a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:11.491 7f8b6e246f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:11.618 7f8b6e2467c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:31:11.618 7f8b72ccc180 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:31:11.618 7f8b72ccc180 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:31:11.618 7f8b6e1f59a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:31:11.618 7f8b6e1f59a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:31:11.628 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:31:11.673 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:31:11.674 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:31:11.674 7f8b72ccc180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:31:20.303 7fd68b48e580 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:31:20.303 7fd68b48e580 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:31:20.303 7fd68b48e580 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:31:20.307 7fd68b48e580 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:31:20.307 7fd686a0e2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:31:20.308 7fd686a0e2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:31:20.308 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:31:20.308 7fd686a0e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:31:20.309 7fd686a0e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:31:20.309 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:31:20.309 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.309 7fd686a0e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:31:20.309 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.309 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.309 7fd686a0e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353080309082 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:31:20.309 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.310 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.310 7fd686a0e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:20.310 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.310 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.310 7fd686a0e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:20.310 7fd686a0e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.311 7fd686a0ee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:20.311 7fd686a0ee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:31:20.312 7fd686a0ec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:31:20.313 7fd6869bd540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.313 7fd6869bd540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:20.313 7fd6869bd540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.313 7fd6869bd540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.313 7fd6869bd540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:20.313 7fd6869bd540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.370 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:31:20.377 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:31:20.378 7fd6869a40e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.378 7fd6869a40e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:20.378 7fd6869a40e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.378 7fd6869a40e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.378 7fd6869a40e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:20.378 7fd6869a40e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.379 7fd6869bdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:20.380 7fd6869bdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:20.381 7fd6869bd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.381 7fd6869bd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:20.381 7fd6869bd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.382 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:31:20.382 7fd6869bd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.383 7fd6869bd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:20.383 7fd6869bd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.392 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:20.392 7fd6869a4c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.392 7fd6869a4c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:20.392 7fd6869a4c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.395 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:20.395 7fd6869a4c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.395 7fd6869a4c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:20.395 7fd6869a4c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.396 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:31:20.396 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:31:20.396 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:31:20.396 7fd6869a4f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:20.397 7fd688084220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.397 7fd688084220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:20.397 7fd688084220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.397 7fd688084220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.397 7fd688084220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:20.397 7fd688084220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.400 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:31:20.403 7fd6869a4f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:20.404 7fd688084ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.404 7fd688084ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:20.404 7fd688084ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.420 7fd686a0e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:31:20.421 7fd688084e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.421 7fd688084e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:20.421 7fd688084e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:20.447 7fd6869bdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:20.447 7fd6869bdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:20.618 7fd6869bdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:20.619 7fd6869bdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:21.054 7fd6869bdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:21.055 7fd6869bdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:21.312 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:21.692 7fd6869bdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:21.693 7fd6869bdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:22.312 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:22.597 7fd6869bdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:22.598 7fd6869bdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:23.313 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:23.965 7fd6869bdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:23.966 7fd6869bdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:24.309 7fd688084e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:24.310 7fd688084e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:24.310 7fd688084e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:24.311 7fd688084e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:24.311 7fd688084e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:24.311 7fd688084e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:24.311 7fd688084e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:24.311 7fd688084e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:31:24.312 7fd688084e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:24.313 7fd688084e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:24.313 7fd688084e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:24.313 7fd688084720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:31:24.314 7fd688084720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:31:24.314 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:25.315 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:25.921 7fd6869bdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:25.922 7fd6869bdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:26.317 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:27.318 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:28.319 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:29.320 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:29.987 7fd6869bdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:29.988 7fd6869bdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:30.321 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:31.322 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:32.323 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:33.325 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:34.326 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:35.327 7fd686a0ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:35.411 7fd6869bdc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:31:35.412 7fd6869bdc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:31:35.412 7fd6871b0c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:35.501 7fd686a0e7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:31:35.501 7fd68b48e580 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:31:35.501 7fd68b48e580 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:31:35.501 7fd6869bd680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:31:35.501 7fd6869bd680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:31:35.508 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:31:35.508 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:31:35.508 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:31:35.508 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:31:35.508 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:31:35.508 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:31:35.508 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:31:35.509 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:31:35.509 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:31:35.509 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:31:35.509 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:31:35.509 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:31:35.509 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:31:35.509 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:31:35.509 7fd68b48e580 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:31:43.846 7f79133f7180 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:31:43.846 7f79133f7180 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:31:43.846 7f79133f7180 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:31:43.849 7f79133f7180 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:31:43.850 7f790e97e2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:31:43.850 7f790e97e2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:31:43.850 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:31:43.850 7f790e97e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:31:43.851 7f790e97e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:31:43.851 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:31:43.851 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.851 7f790e97e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:31:43.851 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.851 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.851 7f790e97e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353103851076 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:31:43.851 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.852 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.852 7f790e97e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:43.852 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.852 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.852 7f790e97e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:43.852 7f790e97e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.853 7f790e97ee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:43.853 7f790e97ee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:31:43.854 7f790e97ec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:31:43.854 7f790e92e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.854 7f790e92e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:43.855 7f790e92e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.855 7f790e92e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.855 7f790e92e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:43.855 7f790e92e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.908 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:31:43.916 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:31:43.917 7f790e8ce0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.917 7f790e8ce0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:43.917 7f790e8ce0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.917 7f790e8ce0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.917 7f790e8ce0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:43.918 7f790e8ce0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.918 7f790e92ec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:43.919 7f790e92ec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:43.920 7f790e92e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.920 7f790e92e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:43.920 7f790e92e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.921 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:31:43.921 7f790e92e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.921 7f790e92e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:43.921 7f790e92e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.930 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:43.931 7f790e8cec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.931 7f790e8cec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:43.931 7f790e8cec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.933 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:43.933 7f790e8cec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.933 7f790e8cec20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:43.933 7f790e8cec20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.934 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:31:43.934 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:31:43.934 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:31:43.934 7f790e8cef40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:43.935 7f790e803220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.935 7f790e803220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:43.935 7f790e803220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.935 7f790e803220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.935 7f790e803220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:43.935 7f790e803220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.937 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:31:43.940 7f790e8cef40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:43.940 7f790e803ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.941 7f790e803ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:43.941 7f790e803ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.956 7f790e97e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:31:43.958 7f790e803e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:43.958 7f790e803e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:43.958 7f790e803e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:44.043 7f790e92ec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:44.044 7f790e92ec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:44.213 7f790e92ec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:44.214 7f790e92ec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:44.663 7f790e92ec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:44.663 7f790e92ec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:44.854 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:45.613 7f790e92ec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:45.614 7f790e92ec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:45.855 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:45.871 7f790e803e00 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:31:46.857 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:47.851 7f790e803e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:47.852 7f790e803e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:47.852 7f790e803e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:47.852 7f790e803e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:47.852 7f790e803e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:47.853 7f790e803e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:47.853 7f790e803e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:47.853 7f790e803e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:31:47.853 7f790e803e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:47.853 7f790e803e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:47.853 7f790e803e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:47.853 7f790e803860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:31:47.854 7f790e803860 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:31:47.858 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:47.874 7f790e92ec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:47.875 7f790e92ec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:48.859 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:49.860 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:50.861 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:51.751 7f790e92ec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:51.751 7f790e92ec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:51.863 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:52.864 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:53.865 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:54.866 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:55.867 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:56.868 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:57.108 7f790e92ec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:31:57.109 7f790e92ec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:57.869 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:58.870 7f790e97ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:58.917 7f790e92ec20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:31:58.917 7f790e92ec20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:31:58.918 7f790e92ec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:59.031 7f790e97e7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:31:59.032 7f79133f7180 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:31:59.032 7f79133f7180 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:31:59.032 7f790e92eb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:31:59.032 7f790e92eb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:31:59.041 7f79133f7180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:32:07.479 7f52aa9e0540 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:32:07.479 7f52aa9e0540 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:32:07.479 7f52aa9e0540 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:32:07.482 7f52aa9e0540 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:32:07.483 7f52a5c402c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:32:07.483 7f52a5c402c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:32:07.483 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:32:07.483 7f52a5c407c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:32:07.484 7f52a5c407c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:32:07.484 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:32:07.484 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.484 7f52a5c407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:32:07.484 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.484 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.484 7f52a5c407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353127484089 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:32:07.484 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.485 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.485 7f52a5c407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:07.485 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.485 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.485 7f52a5c407c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:07.485 7f52a5c407c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.485 7f52a5c40e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:07.486 7f52a5c40e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:32:07.487 7f52a5c40c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:32:07.487 7f52a5bf0540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.487 7f52a5bf0540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:07.487 7f52a5bf0540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.487 7f52a5bf0540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.487 7f52a5bf0540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:07.487 7f52a5bf0540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.541 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:32:07.549 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:32:07.549 7f52a5b51040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.550 7f52a5b51040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:07.550 7f52a5b51040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.550 7f52a5b51040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.550 7f52a5b51040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:07.550 7f52a5b51040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.550 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:07.551 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:07.552 7f52a5bf07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.552 7f52a5bf07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:07.552 7f52a5bf07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.553 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:32:07.553 7f52a5bf07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.553 7f52a5bf07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:07.553 7f52a5bf07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.563 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:07.563 7f52a5b51ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.564 7f52a5b51ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:07.564 7f52a5b51ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.566 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:07.566 7f52a5b51ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.567 7f52a5b51ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:07.567 7f52a5b51ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.567 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:32:07.567 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:32:07.567 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:32:07.567 7f52a5b51e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:07.568 7f52a5ac30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.568 7f52a5ac30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:07.568 7f52a5ac30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.568 7f52a5ac30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.568 7f52a5ac30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:07.568 7f52a5ac30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.571 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:32:07.574 7f52a5b51e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:07.574 7f52a5ac39a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.575 7f52a5ac39a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:07.575 7f52a5ac39a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.593 7f52a5c407c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:32:07.595 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.595 7f52a5ac3cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:07.595 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:07.637 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:07.638 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:07.757 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:07.758 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:07.907 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:07.908 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:08.088 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:08.089 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:08.376 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:08.377 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:08.486 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:08.951 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:08.952 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:09.488 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:10.489 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:10.498 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:10.499 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:11.484 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:11.485 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:11.486 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:11.486 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:11.486 7f52a5ac3cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:11.486 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:11.486 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:11.486 7f52a5ac3cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:32:11.488 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:11.488 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:11.488 7f52a5ac3cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:11.488 7f52a5ac35e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:32:11.489 7f52a5ac35e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:32:11.490 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:12.491 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:13.179 7f52a5ac35e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:32:13.492 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:13.603 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:13.604 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:14.493 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:15.495 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:16.496 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:17.497 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:18.498 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:19.500 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:20.501 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:21.502 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:21.712 7f52a5bf0a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:21.713 7f52a5bf0a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:22.504 7f52a5c40f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:22.516 7f52a5bf0a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:32:22.516 7f52a5bf0a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:32:22.517 7f52a5bf0a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:22.651 7f52a5c407c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:32:22.651 7f52aa9e0540 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:32:22.651 7f52aa9e0540 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:32:22.651 7f52a5bf09a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:32:22.651 7f52a5bf09a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:32:22.658 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:32:22.659 7f52aa9e0540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:32:30.940 7fc945d7adc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:32:30.941 7fc945d7adc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:32:30.941 7fc945d7adc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:32:30.944 7fc945d7adc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:32:30.944 7fc9412fa2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:32:30.945 7fc9412fa2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:32:30.945 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:32:30.945 7fc9412fa7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:32:30.945 7fc9412fa7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:32:30.946 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:32:30.946 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.946 7fc9412fa7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:32:30.946 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.946 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.946 7fc9412fa7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353150945900 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:32:30.946 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.947 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.947 7fc9412fa7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:30.947 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.947 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.947 7fc9412fa7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:30.947 7fc9412fa7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.948 7fc9412fae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:30.948 7fc9412fae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:32:30.949 7fc9412fac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:32:30.949 7fc9412a9540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.950 7fc9412a9540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:30.950 7fc9412a9540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.950 7fc9412a9540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.950 7fc9412a9540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:30.950 7fc9412a9540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.003 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:32:31.010 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:32:31.011 7fc940bee040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.011 7fc940bee040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:31.011 7fc940bee040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.012 7fc940bee040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.012 7fc940bee040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:31.012 7fc940bee040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.012 7fc9412a9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:31.013 7fc9412a9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:31.014 7fc9412a97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.014 7fc9412a97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:31.014 7fc9412a97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.015 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:32:31.015 7fc9412a97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.015 7fc9412a97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:31.015 7fc9412a97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.024 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:31.025 7fc940beeae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.025 7fc940beeae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:31.025 7fc940beeae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.027 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:31.028 7fc940beeae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.028 7fc940beeae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:31.028 7fc940beeae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.028 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:32:31.028 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:32:31.028 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:32:31.029 7fc940beee00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:31.029 7fc94126c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.029 7fc94126c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:31.029 7fc94126c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.030 7fc94126c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.030 7fc94126c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:31.030 7fc94126c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.032 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:32:31.035 7fc940beee00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:31.035 7fc94126c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.036 7fc94126c9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:31.036 7fc94126c9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.051 7fc9412fa7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:32:31.053 7fc94126ccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.053 7fc94126ccc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:31.053 7fc94126ccc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:31.140 7fc9412a9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:31.141 7fc9412a9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:31.375 7fc9412a9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:31.376 7fc9412a9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:31.677 7fc9412a9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:31.678 7fc9412a9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:31.949 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:32.088 7fc9412a9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:32.089 7fc9412a9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:32.950 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:33.066 7fc9412a9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:33.067 7fc9412a9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:33.952 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:34.945 7fc9433f10e0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:34.948 7fc9433f10e0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:34.948 7fc9433f10e0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:34.948 7fc9433f10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:34.948 7fc9433f10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:34.948 7fc9433f10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:34.948 7fc9433f10e0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:34.949 7fc9433f10e0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:32:34.949 7fc9433f10e0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:34.949 7fc9433f10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:34.949 7fc9433f10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:34.949 7fc94126c720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:32:34.950 7fc94126c720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:32:34.953 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:35.024 7fc9412a9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:35.025 7fc9412a9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:35.953 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:36.955 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:37.956 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:38.957 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:39.957 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:40.386 7fc9412a9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:40.386 7fc9412a9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:40.801 7fc941a9cc20 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:32:40.959 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:41.960 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:42.961 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:43.962 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:44.963 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:45.965 7fc9412faf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:46.011 7fc9412a9a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:32:46.011 7fc9412a9a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:32:46.012 7fc9412a9a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:46.107 7fc9412fa7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:32:46.107 7fc945d7adc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:32:46.107 7fc945d7adc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:32:46.107 7fc9412a99a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:32:46.107 7fc9412a99a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:32:46.115 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:32:46.116 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:32:46.116 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:32:46.116 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:32:46.116 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:32:46.116 7fc945d7adc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:32:54.702 7f104b854980 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:32:54.702 7f104b854980 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:32:54.702 7f104b854980 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:32:54.706 7f104b854980 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:32:54.706 7f1046dcd2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:32:54.706 7f1046dcd2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:32:54.707 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:32:54.707 7f1046dcd7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:32:54.707 7f1046dcd7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:32:54.708 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:32:54.708 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.708 7f1046dcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:32:54.708 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.708 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.708 7f1046dcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353174707795 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:32:54.708 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.708 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.709 7f1046dcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:54.709 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.709 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.709 7f1046dcd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:54.709 7f1046dcd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.709 7f1046dcde00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:54.710 7f1046dcde00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:32:54.711 7f1046dcdc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:32:54.711 7f1046d7c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.711 7f1046d7c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:54.711 7f1046d7c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.711 7f1046d7c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.711 7f1046d7c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:54.711 7f1046d7c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.766 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:32:54.773 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:32:54.774 7f1046d22040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.774 7f1046d22040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:54.774 7f1046d22040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.775 7f1046d22040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.775 7f1046d22040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:54.775 7f1046d22040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.775 7f1046d7ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:54.776 7f1046d7ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:54.777 7f1046d7c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.777 7f1046d7c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:54.777 7f1046d7c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.778 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:32:54.778 7f1046d7c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.778 7f1046d7c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:54.778 7f1046d7c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.789 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:54.790 7f1046d22ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.790 7f1046d22ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:54.791 7f1046d22ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.793 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:54.793 7f1046d22ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.793 7f1046d22ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:54.793 7f1046d22ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.794 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:32:54.794 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:32:54.794 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:32:54.794 7f1046d22e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:54.795 7f104837d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.795 7f104837d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:54.795 7f104837d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.795 7f104837d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.795 7f104837d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:54.795 7f104837d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.798 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:32:54.801 7f1046d22e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:54.801 7f104837d9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.801 7f104837d9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:54.801 7f104837d9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.815 7f1046dcd7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:32:54.817 7f104837dcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.817 7f104837dcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:54.817 7f104837dcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:54.873 7f1046d7ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:54.874 7f1046d7ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:55.045 7f1046d7ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:55.074 7f1046d7ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:55.544 7f1046d7ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:55.545 7f1046d7ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:55.711 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:56.253 7f1046d7ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:56.254 7f1046d7ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:56.712 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:57.475 7f1046d7ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:32:57.475 7f1046d7ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:57.713 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:58.707 7f104837dcc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:58.709 7f104837dcc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:58.709 7f104837dcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:58.709 7f104837dcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:58.709 7f104837dcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:58.709 7f104837dcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:58.709 7f104837dcc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:58.709 7f104837dcc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:32:58.711 7f104837dcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:58.711 7f104837dcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:58.711 7f104837dcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:58.711 7f104837d5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:32:58.712 7f104837d5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:32:58.714 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:59.715 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:00.716 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:00.768 7f1046d7ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:00.769 7f1046d7ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:01.717 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:02.719 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:03.720 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:04.721 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:05.722 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:06.723 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:07.599 7f1046d7ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:07.600 7f1046d7ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:07.724 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:08.013 7f104837d5e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:33:08.726 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:09.727 7f1046dcdf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:09.810 7f1046d7ca40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:33:09.811 7f1046d7ca40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:33:09.811 7f1046d7ca40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:09.881 7f1046dcd7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:33:09.881 7f104b854980 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:33:09.881 7f104b854980 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:33:09.881 7f1046d7c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:33:09.881 7f1046d7c9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:33:09.890 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:33:09.891 7f104b854980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:33:18.427 7fcb89186300 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:33:18.427 7fcb89186300 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:33:18.427 7fcb89186300 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:33:18.430 7fcb89186300 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:33:18.430 7fcb846f52c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:33:18.430 7fcb846f52c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:33:18.431 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:33:18.431 7fcb846f57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:33:18.431 7fcb846f57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:33:18.431 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:33:18.432 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.432 7fcb846f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:33:18.432 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.432 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.432 7fcb846f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353198431766 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:33:18.432 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.432 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.433 7fcb846f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:18.433 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.433 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.433 7fcb846f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:18.433 7fcb846f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.433 7fcb846f5e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:18.433 7fcb846f5e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:33:18.434 7fcb846f5c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:33:18.435 7fcb846a4540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.435 7fcb846a4540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:18.435 7fcb846a4540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.435 7fcb846a4540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.435 7fcb846a4540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:18.435 7fcb846a4540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.487 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:33:18.494 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:33:18.495 7fcb8456d040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.495 7fcb8456d040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:18.495 7fcb8456d040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.495 7fcb8456d040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.496 7fcb8456d040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:18.496 7fcb8456d040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.496 7fcb846a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:18.497 7fcb846a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:18.497 7fcb846a47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.498 7fcb846a47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:18.498 7fcb846a47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.499 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:33:18.499 7fcb846a47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.499 7fcb846a47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:18.499 7fcb846a47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.509 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:18.509 7fcb8456dae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.509 7fcb8456dae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:18.509 7fcb8456dae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.512 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:18.512 7fcb8456dae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.512 7fcb8456dae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:18.512 7fcb8456dae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.512 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:33:18.513 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:33:18.513 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:33:18.513 7fcb8456de00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:18.513 7fcb85d8b0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.514 7fcb85d8b0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:18.514 7fcb85d8b0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.514 7fcb85d8b0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.514 7fcb85d8b0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:18.514 7fcb85d8b0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.517 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:33:18.554 7fcb8456de00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:18.577 7fcb85d8bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.578 7fcb85d8bc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:18.578 7fcb85d8bc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.590 7fcb846f57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:33:18.592 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.592 7fcb85d8be00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:18.592 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:18.622 7fcb846a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:18.622 7fcb846a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:18.927 7fcb846a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:18.928 7fcb846a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:19.435 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:19.507 7fcb846a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:19.507 7fcb846a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:20.435 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:20.507 7fcb846a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:20.507 7fcb846a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:21.436 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:22.217 7fcb846a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:22.218 7fcb846a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:22.432 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:22.433 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:22.433 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:22.434 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:22.434 7fcb85d8be00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:22.434 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:22.434 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:22.434 7fcb85d8be00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:33:22.436 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:22.436 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:22.436 7fcb85d8be00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:22.436 7fcb85d8b5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:33:22.437 7fcb85d8b5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:33:22.437 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:23.438 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:24.440 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:24.514 7fcb846a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:24.514 7fcb846a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:25.440 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:26.441 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:27.443 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:27.712 7fcb846a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:27.713 7fcb846a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:28.443 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:29.445 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:30.446 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:31.447 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:32.448 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:33.438 7fcb846a4a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:33:33.439 7fcb846a4a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:33:33.439 7fcb846a4a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:33.449 7fcb846f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:33.625 7fcb846f57c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:33:33.625 7fcb89186300 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:33:33.625 7fcb89186300 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:33:33.625 7fcb846a49a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:33:33.625 7fcb846a49a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:33:33.632 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:33:33.632 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:33:33.632 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:33:33.633 7fcb89186300 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:33:42.502 7fed767daf80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:33:42.502 7fed767daf80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:33:42.502 7fed767daf80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:33:42.505 7fed767daf80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:33:42.506 7fed71d522c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:33:42.506 7fed71d522c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:33:42.506 7fed71d527c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:33:42.506 7fed71d527c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:33:42.507 7fed71d527c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:33:42.507 7fed71d527c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:33:42.507 7fed71d527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.507 7fed71d527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:33:42.507 7fed71d527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.507 7fed71d527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.507 7fed71d527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353222507279 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:33:42.507 7fed71d527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.508 7fed71d527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.508 7fed71d527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:42.508 7fed71d527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.508 7fed71d527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.508 7fed71d527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:42.508 7fed71d527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.509 7fed71d52e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:42.509 7fed71d52e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:33:42.510 7fed71d52c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:33:42.511 7fed71d01540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.511 7fed71d01540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:42.511 7fed71d01540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.511 7fed71d01540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.511 7fed71d01540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:42.511 7fed71d01540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.566 7fed71d527c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:33:42.574 7fed71d527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:33:42.575 7fed71ca5040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.575 7fed71ca5040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:42.575 7fed71ca5040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.575 7fed71ca5040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.576 7fed71ca5040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:42.576 7fed71ca5040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.576 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:42.577 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:42.578 7fed71d017c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.578 7fed71d017c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:42.578 7fed71d017c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.580 7fed71d527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:33:42.580 7fed71d017c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.580 7fed71d017c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:42.580 7fed71d017c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.590 7fed71d527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:42.591 7fed71ca5ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.591 7fed71ca5ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:42.591 7fed71ca5ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.593 7fed71d527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:42.594 7fed71ca5ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.594 7fed71ca5ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:42.594 7fed71ca5ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.594 7fed71d527c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:33:42.594 7fed71d527c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:33:42.595 7fed71d527c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:33:42.595 7fed71ca5e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:42.595 7fed71bd50e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.596 7fed71bd50e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:42.596 7fed71bd50e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.596 7fed71bd50e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.596 7fed71bd50e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:42.596 7fed71bd50e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.598 7fed71d527c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:33:42.602 7fed71ca5e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:42.602 7fed71bd59a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.602 7fed71bd59a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:42.602 7fed71bd59a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.617 7fed71d527c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:33:42.618 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.618 7fed71bd5cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:42.618 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.672 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:42.673 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:42.856 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:42.857 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:43.152 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:43.152 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:43.511 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:43.540 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:43.540 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:44.077 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:44.077 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:44.512 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:45.051 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:45.051 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:45.513 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:46.258 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:46.259 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:46.507 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:46.508 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:46.508 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:46.509 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:46.509 7fed71bd5cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:46.509 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:46.509 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:46.510 7fed71bd5cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:33:46.511 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:46.511 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:46.511 7fed71bd5cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:46.511 7fed71bd55e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:33:46.512 7fed71bd55e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:33:46.514 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:47.515 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:48.516 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:48.952 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:48.952 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:49.546 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:50.547 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:51.549 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:52.550 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:53.551 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:54.553 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:55.554 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:55.832 7fed71d01a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:33:55.833 7fed71d01a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.555 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:57.540 7fed71d01a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:33:57.540 7fed71d01a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:33:57.541 7fed71d01a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:57.556 7fed71d52f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:57.673 7fed71d527c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:33:57.674 7fed767daf80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:33:57.674 7fed767daf80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:33:57.674 7fed71d019a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:33:57.674 7fed71d019a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:33:57.682 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:33:57.682 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:33:57.682 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:33:57.682 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:33:57.682 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:33:57.682 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:33:57.682 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:33:57.682 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:33:57.682 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:33:57.683 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:33:57.683 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:33:57.683 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:33:57.683 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:33:57.683 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:33:57.683 7fed767daf80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:34:06.295 7fe463201700 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:34:06.295 7fe463201700 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:34:06.295 7fe463201700 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:34:06.298 7fe463201700 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:34:06.299 7fe45e5952c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:34:06.299 7fe45e5952c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:34:06.299 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:34:06.299 7fe45e5957c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:06.300 7fe45e5957c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:34:06.300 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:34:06.300 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.300 7fe45e5957c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:34:06.300 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.300 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.300 7fe45e5957c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353246299982 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:34:06.300 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.301 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.301 7fe45e5957c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:06.301 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.302 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.302 7fe45e5957c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:06.302 7fe45e5957c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.302 7fe45e595f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:06.302 7fe45e595f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:34:06.303 7fe45e595c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:34:06.304 7fe45e5457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.304 7fe45e5457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:06.304 7fe45e5457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.304 7fe45e5457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.304 7fe45e5457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:06.304 7fe45e5457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.359 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:34:06.366 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:34:06.367 7fe45e52d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.367 7fe45e52d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:06.368 7fe45e52d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.368 7fe45e52d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.368 7fe45e52d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:06.368 7fe45e52d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.368 7fe45e545c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:06.369 7fe45e545c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:06.370 7fe45e5459a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.370 7fe45e5459a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:06.370 7fe45e5459a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.371 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:34:06.371 7fe45e5459a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.372 7fe45e5459a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:06.372 7fe45e5459a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.381 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:06.382 7fe45e52dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.382 7fe45e52dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:06.382 7fe45e52dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.384 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:06.385 7fe45e52dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.385 7fe45e52dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:06.385 7fe45e52dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.385 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:34:06.385 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:34:06.385 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:34:06.386 7fe45e52df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:06.386 7fe4606a0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.386 7fe4606a0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:06.386 7fe4606a0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.386 7fe4606a0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.386 7fe4606a0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:06.387 7fe4606a0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.389 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:34:06.392 7fe45e52df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:06.392 7fe4606a0ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.392 7fe4606a0ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:06.392 7fe4606a0ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.406 7fe45e5957c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:34:06.407 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.407 7fe4606a0e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:06.408 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:06.505 7fe45e545c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:06.506 7fe45e545c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:06.836 7fe45e545c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:06.837 7fe45e545c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:07.304 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:07.616 7fe45e545c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:07.617 7fe45e545c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:08.305 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:09.306 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:09.531 7fe45e545c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:09.532 7fe45e545c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:10.300 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:10.301 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:10.301 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:10.301 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:10.301 7fe4606a0e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:10.301 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:10.302 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:10.302 7fe4606a0e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:34:10.303 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:10.303 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:10.303 7fe4606a0e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:10.304 7fe4606a0720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:34:10.304 7fe4606a0720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:34:10.307 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:11.308 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:12.309 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:13.310 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:14.246 7fe45e545c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:14.247 7fe45e545c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:14.311 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:15.313 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:16.314 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:17.315 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:18.315 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:19.316 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:20.317 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:21.318 7fe45e5452c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:21.382 7fe45e545c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:34:21.382 7fe45e545c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:34:21.383 7fe45e545c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:21.468 7fe45e5957c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:34:21.468 7fe463201700 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:34:21.468 7fe463201700 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:34:21.468 7fe45e545b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:34:21.468 7fe45e545b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:34:21.475 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:34:21.476 7fe463201700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:34:30.129 7f3122245900 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:34:30.129 7f3122245900 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:34:30.129 7f3122245900 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:34:30.133 7f3122245900 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:34:30.133 7f311d1e32c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:34:30.133 7f311d1e32c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:34:30.134 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:34:30.134 7f311d1e37c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:30.134 7f311d1e37c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:34:30.135 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:34:30.135 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.135 7f311d1e37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:34:30.135 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.135 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.135 7f311d1e37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353270134676 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:34:30.135 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.136 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.136 7f311d1e37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:30.136 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.136 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.136 7f311d1e37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:30.136 7f311d1e37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.137 7f311d1e3e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:30.137 7f311d1e3e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:34:30.138 7f311d1e3c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:34:30.139 7f311d192540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.139 7f311d192540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:30.139 7f311d192540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.139 7f311d192540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.139 7f311d192540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:30.139 7f311d192540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.191 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:34:30.199 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:34:30.200 7f311d0590e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.200 7f311d0590e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:30.200 7f311d0590e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.200 7f311d0590e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.200 7f311d0590e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:30.200 7f311d0590e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.200 7f311d192c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:30.201 7f311d192c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:30.202 7f311d1927c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.202 7f311d1927c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:30.202 7f311d1927c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.203 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:34:30.203 7f311d1927c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.204 7f311d1927c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:30.204 7f311d1927c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.213 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:30.214 7f311d059c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.214 7f311d059c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:30.214 7f311d059c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.216 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:30.217 7f311d059c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.217 7f311d059c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:30.217 7f311d059c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.217 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:34:30.217 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:34:30.218 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:34:30.218 7f311d059f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:30.218 7f311e87a220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.219 7f311e87a220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:30.219 7f311e87a220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.219 7f311e87a220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.219 7f311e87a220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:30.219 7f311e87a220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.221 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:34:30.224 7f311d059f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:30.225 7f311e87aae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.225 7f311e87aae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:30.225 7f311e87aae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.239 7f311d1e37c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:34:30.241 7f311e87ae00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.241 7f311e87ae00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:30.241 7f311e87ae00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:30.336 7f311d192c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:30.337 7f311d192c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:30.672 7f311e87ae00 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:30.697 7f311d192c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:30.698 7f311d192c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:31.138 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:31.360 7f311d192c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:31.361 7f311d192c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:32.139 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:33.069 7f311d192c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:33.070 7f311d192c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:33.141 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:34.135 7f311d059ea0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:34.136 7f311d059ea0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:34.136 7f311d059ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:34.136 7f311d059ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:34.137 7f311d059ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:34.137 7f311d059ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:34.137 7f311d059ea0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:34.137 7f311d059ea0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:34:34.137 7f311d059ea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:34.137 7f311d059ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:34.137 7f311d059ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:34.137 7f311e87a900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:34:34.138 7f311e87a900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:34:34.142 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:35.142 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:36.143 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:37.144 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:37.401 7f311d192c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:37.401 7f311d192c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:38.145 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:39.147 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:40.148 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:41.150 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:42.151 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:43.152 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:44.004 7f311d192c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:44.005 7f311d192c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:44.153 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:45.154 7f311d1e3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:45.210 7f311d192c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:34:45.211 7f311d192c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:34:45.212 7f311d192c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:45.292 7f311d1e37c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:34:45.293 7f3122245900 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:34:45.293 7f3122245900 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:34:45.293 7f311d192b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:34:45.293 7f311d192b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:34:45.300 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:34:45.300 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:34:45.300 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:34:45.300 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:34:45.301 7f3122245900 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:34:54.002 7fbc31be5680 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:34:54.002 7fbc31be5680 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:34:54.002 7fbc31be5680 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:34:54.006 7fbc31be5680 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:34:54.006 7fbc2d15a2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:34:54.006 7fbc2d15a2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:34:54.006 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:34:54.007 7fbc2d15a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:54.007 7fbc2d15a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:34:54.007 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:34:54.007 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.007 7fbc2d15a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:34:54.008 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.008 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.008 7fbc2d15a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353294007466 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:34:54.008 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.008 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.008 7fbc2d15a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:54.008 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.009 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.009 7fbc2d15a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:54.009 7fbc2d15a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.009 7fbc2d15ae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:54.009 7fbc2d15ae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:34:54.010 7fbc2d15ac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:34:54.011 7fbc2d109540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.011 7fbc2d109540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:54.011 7fbc2d109540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.011 7fbc2d109540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.011 7fbc2d109540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:54.011 7fbc2d109540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.065 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:34:54.073 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:34:54.074 7fbc2cfd1040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.074 7fbc2cfd1040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:54.074 7fbc2cfd1040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.074 7fbc2cfd1040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.074 7fbc2cfd1040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:54.074 7fbc2cfd1040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.074 7fbc2d109a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:54.075 7fbc2d109a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:54.076 7fbc2d1097c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.076 7fbc2d1097c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:54.076 7fbc2d1097c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.077 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:34:54.078 7fbc2d1097c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.078 7fbc2d1097c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:54.078 7fbc2d1097c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.088 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:54.089 7fbc2cfd1ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.089 7fbc2cfd1ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:54.089 7fbc2cfd1ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.091 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:54.092 7fbc2cfd1ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.092 7fbc2cfd1ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:54.092 7fbc2cfd1ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.092 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:34:54.093 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:34:54.093 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:34:54.093 7fbc2cfd1e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:54.093 7fbc2e7f00e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.094 7fbc2e7f00e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:54.094 7fbc2e7f00e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.094 7fbc2e7f00e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.094 7fbc2e7f00e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:54.094 7fbc2e7f00e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.096 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:34:54.099 7fbc2cfd1e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:54.099 7fbc2e7f09a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.100 7fbc2e7f09a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:54.100 7fbc2e7f09a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.115 7fbc2d15a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:34:54.116 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.116 7fbc2e7f0cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:54.116 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:54.195 7fbc2d109a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:54.196 7fbc2d109a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:54.409 7fbc2d109a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:54.410 7fbc2d109a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:54.873 7fbc2d109a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:54.874 7fbc2d109a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:55.010 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:55.431 7fbc2d109a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:55.432 7fbc2d109a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:56.012 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:56.102 7fbc2d109a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:56.103 7fbc2d109a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:57.013 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:57.888 7fbc2e7f0cc0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:57.920 7fbc2d109a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:34:57.921 7fbc2d109a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:58.007 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:58.008 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:58.009 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:58.009 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:58.009 7fbc2e7f0cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:58.009 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:58.009 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:58.009 7fbc2e7f0cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:34:58.010 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:58.010 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:58.010 7fbc2e7f0cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:58.010 7fbc2e7f0720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:34:58.010 7fbc2e7f0720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:34:58.014 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:59.016 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:00.017 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:01.018 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:02.020 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:02.880 7fbc2d109a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:02.881 7fbc2d109a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:03.021 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:04.022 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:05.023 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:06.024 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:07.026 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:08.027 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:09.028 7fbc2d15af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:09.110 7fbc2d109a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:35:09.111 7fbc2d109a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:35:09.111 7fbc2d109a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:09.197 7fbc2d15a7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:35:09.197 7fbc31be5680 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:35:09.197 7fbc31be5680 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:35:09.197 7fbc2d1099a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:35:09.197 7fbc2d1099a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:35:09.204 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:35:09.205 7fbc31be5680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:35:17.527 7f697f10bbc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:35:17.527 7f697f10bbc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:35:17.527 7f697f10bbc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:35:17.530 7f697f10bbc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:35:17.531 7f697a3432c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:35:17.531 7f697a3432c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:35:17.531 7f697a3437c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:35:17.531 7f697a3437c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:35:17.532 7f697a3437c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:35:17.532 7f697a3437c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:35:17.532 7f697a3437c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.532 7f697a3437c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:35:17.532 7f697a3437c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.532 7f697a3437c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.532 7f697a3437c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353317532233 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:35:17.532 7f697a3437c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.533 7f697a3437c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.533 7f697a3437c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:17.533 7f697a3437c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.533 7f697a3437c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.534 7f697a3437c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:17.534 7f697a3437c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.534 7f697a343e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:17.534 7f697a343e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:35:17.535 7f697a343c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:35:17.536 7f697a2f2540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.536 7f697a2f2540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:17.536 7f697a2f2540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.536 7f697a2f2540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.536 7f697a2f2540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:17.536 7f697a2f2540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.587 7f697a3437c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:35:17.595 7f697a3437c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:35:17.595 7f697c3a20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.596 7f697c3a20e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:17.596 7f697c3a20e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.596 7f697c3a20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.596 7f697c3a20e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:17.596 7f697c3a20e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.596 7f697a2f2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:17.597 7f697a2f2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:17.599 7f697a2f27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.599 7f697a2f27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:17.599 7f697a2f27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.600 7f697a3437c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:35:17.600 7f697a2f27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.600 7f697a2f27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:17.600 7f697a2f27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.610 7f697a3437c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:17.610 7f697c3a2c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.611 7f697c3a2c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:17.611 7f697c3a2c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.613 7f697a3437c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:17.613 7f697c3a2c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.613 7f697c3a2c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:17.614 7f697c3a2c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.614 7f697a3437c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:35:17.614 7f697a3437c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:35:17.614 7f697a3437c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:35:17.615 7f697c3a2f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:17.615 7f697b8f3220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.615 7f697b8f3220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:17.615 7f697b8f3220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.615 7f697b8f3220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.615 7f697b8f3220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:17.616 7f697b8f3220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.618 7f697a3437c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:35:17.621 7f697c3a2f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:17.621 7f697b8f3ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.621 7f697b8f3ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:17.621 7f697b8f3ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.638 7f697a3437c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:35:17.640 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.640 7f697b8f3e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:17.641 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:17.722 7f697a2f2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:17.725 7f697a2f2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:17.963 7f697a2f2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:17.964 7f697a2f2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:18.396 7f697a2f2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:18.397 7f697a2f2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:18.535 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:19.332 7f697a2f2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:19.332 7f697a2f2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:19.536 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:20.537 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:21.417 7f697a2f2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:21.418 7f697a2f2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:21.533 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:21.534 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:21.534 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:21.534 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:21.534 7f697b8f3e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:21.534 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:21.534 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:21.534 7f697b8f3e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:35:21.536 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:21.536 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:21.536 7f697b8f3e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:21.536 7f697b8f3720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:35:21.537 7f697b8f3720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:35:21.539 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:22.540 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:23.541 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:24.542 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:25.543 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:25.581 7f697a2f2c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:25.582 7f697a2f2c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:25.689 7f697a2f2b80 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:35:26.544 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:27.545 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:28.546 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:29.547 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:30.548 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:31.550 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:32.551 7f697a343f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:32.615 7f697a2f2c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:35:32.615 7f697a2f2c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:35:32.616 7f697a2f2c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:32.656 7f697a3437c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:35:32.656 7f697f10bbc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:35:32.657 7f697f10bbc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:35:32.657 7f697a2f2a40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:35:32.657 7f697a2f2a40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:35:32.670 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:35:32.670 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:35:32.670 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:35:32.670 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:35:32.670 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:35:32.670 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:35:32.670 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:35:32.670 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:35:32.671 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:35:32.671 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:35:32.671 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:35:32.671 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:35:32.671 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:35:32.671 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:35:32.671 7f697f10bbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:35:41.236 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:35:41.236 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:35:41.236 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:35:41.239 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:35:41.240 7fc6a75932c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:35:41.240 7fc6a75932c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:35:41.240 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:35:41.240 7fc6a75937c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:35:41.241 7fc6a75937c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:35:41.241 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:35:41.241 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.241 7fc6a75937c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:35:41.241 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.241 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.241 7fc6a75937c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353341241054 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:35:41.241 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.242 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.242 7fc6a75937c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:41.242 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.242 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.243 7fc6a75937c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:41.243 7fc6a75937c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.243 7fc6a7593f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:41.243 7fc6a7593f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:35:41.244 7fc6a7593c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:35:41.245 7fc6a75427c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.245 7fc6a75427c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:41.245 7fc6a75427c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.245 7fc6a75427c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.245 7fc6a75427c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:41.245 7fc6a75427c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.296 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:35:41.303 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:35:41.304 7fc6a6e750e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.305 7fc6a6e750e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:41.305 7fc6a6e750e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.305 7fc6a6e750e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.305 7fc6a6e750e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:41.305 7fc6a6e750e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.305 7fc6a7542c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:41.306 7fc6a7542c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:41.307 7fc6a75429a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.307 7fc6a75429a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:41.307 7fc6a75429a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.308 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:35:41.308 7fc6a75429a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.309 7fc6a75429a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:41.309 7fc6a75429a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.318 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:41.318 7fc6a6e75c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.319 7fc6a6e75c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:41.319 7fc6a6e75c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.321 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:41.321 7fc6a6e75c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.321 7fc6a6e75c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:41.321 7fc6a6e75c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.322 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:35:41.322 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:35:41.322 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:35:41.322 7fc6a6e75f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:41.323 7fc6a7416220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.323 7fc6a7416220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:41.323 7fc6a7416220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.323 7fc6a7416220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.323 7fc6a7416220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:41.323 7fc6a7416220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.325 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:35:41.329 7fc6a6e75f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:41.329 7fc6a7416ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.329 7fc6a7416ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:41.329 7fc6a7416ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.343 7fc6a75937c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:35:41.344 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.345 7fc6a7416e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:41.345 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:41.431 7fc6a7542c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:41.434 7fc6a7542c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:41.753 7fc6a7542c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:41.754 7fc6a7542c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:42.181 7fc6a7542c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:42.182 7fc6a7542c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:42.244 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:43.209 7fc6a7542c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:43.210 7fc6a7542c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:43.245 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:44.247 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:45.207 7fc6a7542c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:45.208 7fc6a7542c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:45.241 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:45.242 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:45.243 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:45.243 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:45.243 7fc6a7416e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:45.243 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:45.243 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:45.243 7fc6a7416e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:35:45.245 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:45.245 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:45.245 7fc6a7416e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:45.246 7fc6a7416720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:35:45.247 7fc6a7416720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:35:45.247 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:46.248 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:47.249 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:48.250 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:49.251 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:50.196 7fc6a7542c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:35:50.197 7fc6a7542c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:50.252 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:51.253 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:52.255 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:52.959 7fc6a7d35c20 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:35:53.256 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:54.256 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:55.258 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:56.259 7fc6a75422c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:56.323 7fc6a7542c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:35:56.324 7fc6a7542c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:35:56.324 7fc6a7542c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:56.414 7fc6a75937c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:35:56.415 7fc6ac5f3980 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:35:56.415 7fc6ac5f3980 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:35:56.415 7fc6a7542b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:35:56.415 7fc6a7542b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:35:56.423 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:35:56.423 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:35:56.423 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:35:56.423 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:35:56.423 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:35:56.423 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:35:56.423 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:35:56.423 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:35:56.423 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:35:56.424 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:35:56.424 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:35:56.424 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:35:56.424 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:35:56.424 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:35:56.424 7fc6ac5f3980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:36:05.163 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:36:05.163 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:36:05.163 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:36:05.167 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:36:05.167 7f9c59a4b2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:36:05.168 7f9c59a4b2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:36:05.168 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:36:05.168 7f9c59a4b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:36:05.168 7f9c59a4b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:36:05.169 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:36:05.169 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.169 7f9c59a4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:36:05.169 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.169 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.169 7f9c59a4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353365168898 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:36:05.169 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.170 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.170 7f9c59a4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:05.170 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.170 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.170 7f9c59a4b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:05.170 7f9c59a4b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.171 7f9c59a4be00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:05.171 7f9c59a4be00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:36:05.172 7f9c59a4bc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:36:05.172 7f9c599fb540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.172 7f9c599fb540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:05.173 7f9c599fb540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.173 7f9c599fb540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.173 7f9c599fb540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:05.173 7f9c599fb540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.230 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:36:05.239 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:36:05.240 7f9c5995d040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.240 7f9c5995d040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:05.240 7f9c5995d040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.240 7f9c5995d040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.240 7f9c5995d040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:05.240 7f9c5995d040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.241 7f9c599fba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:05.242 7f9c599fba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:05.242 7f9c599fb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.242 7f9c599fb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:05.243 7f9c599fb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.244 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:36:05.244 7f9c599fb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.244 7f9c599fb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:05.244 7f9c599fb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.255 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:05.255 7f9c5995dae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.256 7f9c5995dae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:05.256 7f9c5995dae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.258 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:05.258 7f9c5995dae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.259 7f9c5995dae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:05.259 7f9c5995dae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.259 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:36:05.259 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:36:05.259 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:36:05.259 7f9c5995de00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:05.260 7f9c598cf0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.260 7f9c598cf0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:05.260 7f9c598cf0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.260 7f9c598cf0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.260 7f9c598cf0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:05.260 7f9c598cf0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.263 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:36:05.266 7f9c5995de00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:05.267 7f9c598cf9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.267 7f9c598cf9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:05.267 7f9c598cf9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.281 7f9c59a4b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:36:05.283 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.283 7f9c598cfcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:05.283 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:05.348 7f9c599fba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:05.349 7f9c599fba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:05.484 7f9c599fba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:05.484 7f9c599fba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:05.700 7f9c599fba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:05.701 7f9c599fba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:06.171 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:06.273 7f9c599fba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:06.274 7f9c599fba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:07.172 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:07.527 7f9c599fba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:07.528 7f9c599fba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:08.174 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:09.169 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:09.171 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:09.171 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:09.171 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:09.171 7f9c598cfcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:09.171 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:09.171 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:09.171 7f9c598cfcc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:36:09.173 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:09.173 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:09.173 7f9c598cfcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:09.173 7f9c598cf5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:36:09.174 7f9c598cf5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:36:09.175 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:10.176 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:10.346 7f9c599fba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:10.347 7f9c599fba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:11.177 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:12.178 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:13.178 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:14.180 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:15.181 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:16.183 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:17.184 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:17.245 7f9c599fba40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:17.246 7f9c599fba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:18.185 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:19.186 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:20.188 7f9c59a4bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:20.259 7f9c599fba40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:36:20.259 7f9c599fba40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:36:20.260 7f9c599fba40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:20.334 7f9c599fba40 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:36:20.371 7f9c59a4b7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:36:20.371 7f9c5e5bff80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:36:20.371 7f9c5e5bff80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:36:20.371 7f9c599fb9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:36:20.371 7f9c599fb9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:36:20.379 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:36:20.379 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:36:20.379 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:36:20.379 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:36:20.379 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:36:20.379 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:36:20.380 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:36:20.380 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:36:20.380 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:36:20.380 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:36:20.380 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:36:20.380 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:36:20.380 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:36:20.380 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:36:20.380 7f9c5e5bff80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:36:28.887 7f8d1592e800 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:36:28.887 7f8d1592e800 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:36:28.888 7f8d1592e800 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:36:28.891 7f8d1592e800 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:36:28.891 7f8d10ea82c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:36:28.892 7f8d10ea82c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:36:28.892 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:36:28.892 7f8d10ea87c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:36:28.893 7f8d10ea87c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:36:28.893 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:36:28.893 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.893 7f8d10ea87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:36:28.893 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.893 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.893 7f8d10ea87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353388892953 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:36:28.893 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.894 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.894 7f8d10ea87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:28.895 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.895 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.895 7f8d10ea87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:28.895 7f8d10ea87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.895 7f8d10ea8f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:28.895 7f8d10ea8f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:36:28.896 7f8d10ea8c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:36:28.897 7f8d10e587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.897 7f8d10e587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:28.897 7f8d10e587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.897 7f8d10e587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.897 7f8d10e587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:28.897 7f8d10e587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.952 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:36:28.960 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:36:28.961 7f8d10e3b0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.961 7f8d10e3b0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:28.962 7f8d10e3b0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.962 7f8d10e3b0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.962 7f8d10e3b0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:28.962 7f8d10e3b0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.962 7f8d10e58c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:28.963 7f8d10e58c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:28.964 7f8d10e589a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.964 7f8d10e589a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:28.964 7f8d10e589a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.965 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:36:28.965 7f8d10e589a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.966 7f8d10e589a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:28.966 7f8d10e589a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.980 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:28.980 7f8d10e3bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.981 7f8d10e3bc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:28.981 7f8d10e3bc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.984 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:28.984 7f8d10e3bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.985 7f8d10e3bc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:28.985 7f8d10e3bc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.985 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:36:28.985 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:36:28.986 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:36:28.986 7f8d10e3bf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:28.987 7f8d10d2b220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.987 7f8d10d2b220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:28.987 7f8d10d2b220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.987 7f8d10d2b220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.987 7f8d10d2b220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:28.987 7f8d10d2b220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.991 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:36:28.996 7f8d10e3bf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:28.996 7f8d10d2bae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.997 7f8d10d2bae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:28.997 7f8d10d2bae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:29.013 7f8d10ea87c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:36:29.015 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:29.015 7f8d10d2bcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:29.016 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:29.092 7f8d10e58c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:29.093 7f8d10e58c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:29.321 7f8d10e58c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:29.322 7f8d10e58c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:29.641 7f8d10e58c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:29.641 7f8d10e58c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:29.896 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:30.516 7f8d10e58c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:30.517 7f8d10e58c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:30.898 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:31.900 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:32.128 7f8d10e58c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:32.129 7f8d10e58c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:32.893 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:32.899 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:32.900 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:32.900 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:32.900 7f8d10d2bcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:32.900 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:32.900 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:32.900 7f8d10d2bcc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:36:32.902 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:32.902 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:32.902 7f8d10d2bcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:32.902 7f8d10d2b720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:36:32.904 7f8d10d2b720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:36:32.904 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:33.905 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:34.906 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:35.907 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:36.443 7f8d10e58c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:36.444 7f8d10e58c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:36.908 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:37.910 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:38.911 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:39.912 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:40.913 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:41.915 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:42.916 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:43.918 7f8d10e582c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:43.974 7f8d10e58c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:36:43.974 7f8d10e58c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:36:43.975 7f8d10e58c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:44.081 7f8d10ea87c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:36:44.081 7f8d1592e800 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:36:44.081 7f8d1592e800 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:36:44.082 7f8d10e58b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:36:44.082 7f8d10e58b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:36:44.089 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:36:44.089 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:36:44.090 7f8d1592e800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:36:52.479 7f15bf5072c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:36:52.479 7f15bf5072c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:36:52.479 7f15bf5072c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:36:52.482 7f15bf5072c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:36:52.482 7f15baa8a2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:36:52.483 7f15baa8a2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:36:52.483 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:36:52.483 7f15baa8a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:36:52.483 7f15baa8a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:36:52.483 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:36:52.484 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.484 7f15baa8a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:36:52.484 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.484 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.484 7f15baa8a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353412483770 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:36:52.484 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.485 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.485 7f15baa8a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:52.485 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.485 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.485 7f15baa8a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:52.485 7f15baa8a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.485 7f15baa8ae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:52.486 7f15baa8ae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:36:52.487 7f15baa8ac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:36:52.487 7f15baa39540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.487 7f15baa39540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:52.487 7f15baa39540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.487 7f15baa39540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.487 7f15baa39540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:52.487 7f15baa39540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.539 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:36:52.547 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:36:52.548 7f15bd076040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.548 7f15bd076040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:52.548 7f15bd076040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.548 7f15bd076040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.548 7f15bd076040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:52.548 7f15bd076040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.548 7f15baa39a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:52.550 7f15baa39a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:52.551 7f15baa397c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.551 7f15baa397c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:52.551 7f15baa397c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.552 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:36:52.552 7f15baa397c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.553 7f15baa397c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:52.553 7f15baa397c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.564 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:52.565 7f15bd076ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.565 7f15bd076ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:52.565 7f15bd076ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.568 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:52.568 7f15bd076ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.568 7f15bd076ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:52.569 7f15bd076ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.569 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:36:52.569 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:36:52.569 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:36:52.569 7f15bd076e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:52.570 7f15ba9100e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.570 7f15ba9100e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:52.570 7f15ba9100e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.570 7f15ba9100e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.570 7f15ba9100e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:52.570 7f15ba9100e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.573 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:36:52.576 7f15bd076e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:52.577 7f15ba9109a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.577 7f15ba9109a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:52.577 7f15ba9109a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.592 7f15baa8a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:36:52.594 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.594 7f15ba910cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:52.604 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:52.649 7f15baa39a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:52.650 7f15baa39a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:52.845 7f15baa39a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:52.846 7f15baa39a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:53.162 7f15baa39a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:53.163 7f15baa39a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:53.487 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:53.926 7f15baa39a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:53.927 7f15baa39a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:54.489 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:55.490 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:55.976 7f15baa39a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:36:55.977 7f15baa39a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:56.483 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:56.491 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:56.495 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:56.496 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:56.496 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:56.496 7f15ba910cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:56.496 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:56.496 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:56.496 7f15ba910cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:36:56.496 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:56.497 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:56.497 7f15ba910cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:56.497 7f15ba910720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:36:56.498 7f15ba910720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:36:57.494 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:58.495 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:59.496 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:00.496 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:01.498 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:01.666 7f15baa39a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:01.667 7f15baa39a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:02.499 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:03.500 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:04.501 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:05.502 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:06.503 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:07.496 7f15baa39a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:37:07.496 7f15baa39a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:37:07.497 7f15baa39a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:07.504 7f15baa8af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:07.666 7f15baa8a7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:37:07.666 7f15bf5072c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:37:07.666 7f15bf5072c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:37:07.666 7f15baa399a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:37:07.666 7f15baa399a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:37:07.674 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:37:07.675 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:37:07.675 7f15bf5072c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:37:16.209 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:37:16.209 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:37:16.209 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:37:16.213 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:37:16.213 7f31eda2f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:37:16.214 7f31eda2f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:37:16.214 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:37:16.214 7f31eda2f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:37:16.214 7f31eda2f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:37:16.215 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:37:16.215 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.215 7f31eda2f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:37:16.215 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.215 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.215 7f31eda2f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353436214896 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:37:16.215 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.216 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.216 7f31eda2f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:16.216 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.216 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.216 7f31eda2f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.216 7f31eda2f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.217 7f31eda2fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:16.217 7f31eda2fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:37:16.218 7f31eda2fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:37:16.219 7f31ed9de540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.219 7f31ed9de540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:16.219 7f31ed9de540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.219 7f31ed9de540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.219 7f31ed9de540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.219 7f31ed9de540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.275 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:37:16.283 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:37:16.284 7f31ef85d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.284 7f31ef85d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:16.285 7f31ef85d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.285 7f31ef85d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.285 7f31ef85d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:16.285 7f31ef85d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.285 7f31ed9dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:16.286 7f31ed9dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.287 7f31ed9de7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.287 7f31ed9de7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:16.287 7f31ed9de7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.288 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:37:16.289 7f31ed9de7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.289 7f31ed9de7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.289 7f31ed9de7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.299 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:16.300 7f31ef85dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.300 7f31ef85dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:16.300 7f31ef85dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.302 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:16.302 7f31ef85dc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.303 7f31ef85dc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:16.303 7f31ef85dc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.303 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:37:16.303 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:37:16.303 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:37:16.303 7f31ef85df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:16.304 7f31ed8b1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.304 7f31ed8b1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:16.304 7f31ed8b1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.304 7f31ed8b1220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.304 7f31ed8b1220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:16.304 7f31ed8b1220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.307 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:37:16.310 7f31ef85df40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:16.311 7f31ed8b1ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.311 7f31ed8b1ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:16.311 7f31ed8b1ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.325 7f31eda2f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:37:16.326 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.327 7f31ed8b1e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:16.327 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:16.396 7f31ed9dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:16.397 7f31ed9dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:16.679 7f31ed9dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:16.680 7f31ed9dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:17.086 7f31ed9dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:17.087 7f31ed9dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:17.218 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:18.060 7f31ed9dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:18.061 7f31ed9dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:18.219 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:19.220 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:20.215 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:20.216 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:20.216 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:20.216 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:20.216 7f31ed8b1e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:20.216 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:20.216 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:20.217 7f31ed8b1e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:37:20.218 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:20.218 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:20.218 7f31ed8b1e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:20.218 7f31ed8b1720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:37:20.219 7f31ed8b1720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:37:20.221 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:20.638 7f31ed9dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:20.639 7f31ed9dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:21.222 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:22.223 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:23.225 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:24.226 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:25.227 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:26.229 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:26.765 7f31ed9dec20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:26.766 7f31ed9dec20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:27.230 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:28.231 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:29.232 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:30.233 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:31.235 7f31eda2ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:31.284 7f31ed9dec20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:37:31.285 7f31ed9dec20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:37:31.285 7f31ed9dec20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:31.389 7f31eda2f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:37:31.389 7f31f24bcfc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:37:31.389 7f31f24bcfc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:37:31.389 7f31ed9deb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:37:31.389 7f31ed9deb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:37:31.399 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:37:31.400 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:37:31.400 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:37:31.400 7f31f24bcfc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:37:39.625 7f1745fe5100 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:37:39.625 7f1745fe5100 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:37:39.625 7f1745fe5100 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:37:39.628 7f1745fe5100 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:37:39.628 7f174156e2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:37:39.628 7f174156e2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:37:39.629 7f174156e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:37:39.629 7f174156e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:37:39.629 7f174156e7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:37:39.629 7f174156e7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:37:39.629 7f174156e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.630 7f174156e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:37:39.630 7f174156e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.630 7f174156e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.630 7f174156e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353459629633 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:37:39.630 7f174156e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.630 7f174156e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.630 7f174156e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:39.631 7f174156e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.631 7f174156e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.631 7f174156e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:39.631 7f174156e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.631 7f174156ee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:39.631 7f174156ee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:37:39.632 7f174156ec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:37:39.632 7f174151e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.633 7f174151e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:39.633 7f174151e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.633 7f174151e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.634 7f174151e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:39.634 7f174151e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.685 7f174156e7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:37:39.692 7f174156e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:37:39.693 7f17414bc040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.693 7f17414bc040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:39.694 7f17414bc040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.694 7f17414bc040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.694 7f17414bc040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:39.694 7f17414bc040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.694 7f174151ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:39.695 7f174151ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:39.695 7f174151e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.696 7f174151e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:39.696 7f174151e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.697 7f174156e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:37:39.697 7f174151e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.697 7f174151e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:39.697 7f174151e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.707 7f174156e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:39.707 7f17414bcae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.708 7f17414bcae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:39.708 7f17414bcae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.710 7f174156e7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:39.711 7f17414bcae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.711 7f17414bcae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:39.711 7f17414bcae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.711 7f174156e7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:37:39.712 7f174156e7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:37:39.712 7f174156e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:37:39.712 7f17414bce00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:39.713 7f17413f10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.713 7f17413f10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:39.713 7f17413f10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.713 7f17413f10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.713 7f17413f10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:39.713 7f17413f10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.716 7f174156e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:37:39.719 7f17414bce00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:39.719 7f17413f19a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.719 7f17413f19a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:39.719 7f17413f19a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.734 7f174156e7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:37:39.735 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.736 7f17413f1cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:39.736 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:39.807 7f174151ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:39.808 7f174151ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:39.989 7f174151ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:39.990 7f174151ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.331 7f174151ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:40.331 7f174151ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:40.632 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:41.123 7f174151ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:41.124 7f174151ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:41.633 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:42.635 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:42.751 7f174151ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:42.751 7f174151ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:43.059 7f17413f1cc0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:37:43.630 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:43.631 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:43.631 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:43.631 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:43.632 7f17413f1cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:43.632 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:43.632 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:43.632 7f17413f1cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:37:43.632 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:43.632 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:43.632 7f17413f1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:43.633 7f17413f1720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:37:43.634 7f17413f1720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:37:43.635 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:44.636 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:45.638 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:46.639 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:46.919 7f174151ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:46.920 7f174151ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:47.640 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:48.642 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:49.643 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:50.644 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:51.645 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:52.504 7f174151ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:37:52.505 7f174151ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:52.646 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:53.647 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:54.648 7f174156ef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:54.715 7f174151ea40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:37:54.715 7f174151ea40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:37:54.716 7f174151ea40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:54.812 7f174156e7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:37:54.812 7f1745fe5100 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:37:54.813 7f1745fe5100 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:37:54.813 7f174151e9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:37:54.813 7f174151e9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:37:54.821 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:37:54.822 7f1745fe5100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:38:03.385 7f142473ed80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:38:03.385 7f142473ed80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:38:03.385 7f142473ed80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:38:03.388 7f142473ed80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:38:03.389 7f141fcb42c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:38:03.389 7f141fcb42c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:38:03.389 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:38:03.389 7f141fcb47c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:03.390 7f141fcb47c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:38:03.390 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:38:03.390 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.390 7f141fcb47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:38:03.390 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.390 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.390 7f141fcb47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353483390214 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:38:03.390 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.391 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.391 7f141fcb47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:03.391 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.391 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.392 7f141fcb47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:03.392 7f141fcb47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.392 7f141fcb4e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:03.392 7f141fcb4e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:38:03.393 7f141fcb4c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:38:03.394 7f141fc64540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.394 7f141fc64540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:03.394 7f141fc64540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.394 7f141fc64540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.394 7f141fc64540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:03.394 7f141fc64540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.449 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:38:03.457 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:38:03.458 7f141fb2a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.459 7f141fb2a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:03.459 7f141fb2a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.459 7f141fb2a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.459 7f141fb2a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:03.459 7f141fb2a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.459 7f141fc64c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:03.460 7f141fc64c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:03.461 7f141fc647c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.461 7f141fc647c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:03.461 7f141fc647c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.462 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:38:03.463 7f141fc647c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.463 7f141fc647c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:03.463 7f141fc647c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.473 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:03.473 7f141fb2ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.474 7f141fb2ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:03.474 7f141fb2ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.477 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:03.477 7f141fb2ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.477 7f141fb2ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:03.477 7f141fb2ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.478 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:38:03.478 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:38:03.478 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:38:03.478 7f141fb2af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:03.479 7f142128c220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.479 7f142128c220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:03.479 7f142128c220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.479 7f142128c220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.479 7f142128c220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:03.479 7f142128c220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.482 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:38:03.486 7f141fb2af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:03.486 7f142128cae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.486 7f142128cae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:03.486 7f142128cae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.500 7f141fcb47c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:38:03.502 7f142128ce00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.502 7f142128ce00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:03.502 7f142128ce00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:03.583 7f141fc64c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:03.584 7f141fc64c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:03.899 7f141fc64c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:03.900 7f141fc64c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:04.312 7f141fc64c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:04.313 7f141fc64c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:04.393 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:05.356 7f141fc64c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:05.357 7f141fc64c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:05.394 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:06.395 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:07.390 7f142128ce00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:07.391 7f142128ce00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:07.391 7f142128ce00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:07.391 7f142128ce00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:07.392 7f142128ce00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:07.392 7f142128ce00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:07.392 7f142128ce00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:07.392 7f142128ce00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:38:07.394 7f142128ce00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:07.394 7f142128ce00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:07.394 7f142128ce00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:07.394 7f142128c720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:38:07.395 7f142128c720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:38:07.396 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:07.603 7f141fc64c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:07.604 7f141fc64c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:08.397 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:09.398 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:10.399 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:10.544 7f141fc64c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:10.545 7f141fc64c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:10.784 7f1420456c20 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:11.400 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:12.402 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:13.402 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:14.403 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:15.405 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:16.406 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:17.408 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:18.191 7f141fc64c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:18.192 7f141fc64c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:18.410 7f141fcb4f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:18.493 7f141fc64c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:38:18.494 7f141fc64c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:38:18.495 7f141fc64c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:18.604 7f141fcb47c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:38:18.604 7f142473ed80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:38:18.605 7f142473ed80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:38:18.605 7f141fc64a40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:38:18.605 7f141fc64a40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:38:18.612 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:38:18.612 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:38:18.612 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:38:18.612 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:38:18.612 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:38:18.612 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:38:18.613 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:38:18.613 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:38:18.613 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:38:18.613 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:38:18.613 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:38:18.613 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:38:18.613 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:38:18.613 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:38:18.613 7f142473ed80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:38:26.976 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:38:26.976 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:38:26.976 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:38:26.980 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:38:26.981 7f8bc49652c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:38:26.981 7f8bc49652c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:38:26.981 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:38:26.981 7f8bc49657c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:26.982 7f8bc49657c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:38:26.982 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:38:26.982 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.982 7f8bc49657c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:38:26.982 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.982 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.982 7f8bc49657c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353506982178 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:38:26.982 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.983 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.983 7f8bc49657c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:26.983 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.983 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.984 7f8bc49657c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:26.984 7f8bc49657c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.984 7f8bc4965e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:26.984 7f8bc4965e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:38:26.985 7f8bc4965c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:38:26.986 7f8bc4914540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.986 7f8bc4914540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:26.986 7f8bc4914540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.986 7f8bc4914540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:26.986 7f8bc4914540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:26.986 7f8bc4914540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.041 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:38:27.048 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:38:27.049 7f8bc48b8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.049 7f8bc48b8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:27.049 7f8bc48b8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.049 7f8bc48b8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.049 7f8bc48b8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:27.049 7f8bc48b8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.050 7f8bc4914a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:27.051 7f8bc4914a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:27.052 7f8bc49147c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.052 7f8bc49147c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:27.052 7f8bc49147c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.053 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:38:27.054 7f8bc49147c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.054 7f8bc49147c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:27.054 7f8bc49147c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.064 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:27.064 7f8bc48b8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.065 7f8bc48b8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:27.065 7f8bc48b8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.067 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:27.067 7f8bc48b8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.067 7f8bc48b8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:27.067 7f8bc48b8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.068 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:38:27.068 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:38:27.068 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:38:27.068 7f8bc48b8e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:27.068 7f8bc47eb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.069 7f8bc47eb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:27.069 7f8bc47eb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.069 7f8bc47eb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.069 7f8bc47eb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:27.069 7f8bc47eb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.071 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:38:27.075 7f8bc48b8e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:27.075 7f8bc47eb9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.075 7f8bc47eb9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:27.075 7f8bc47eb9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.089 7f8bc49657c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:38:27.091 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.091 7f8bc47ebb80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:27.091 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:27.150 7f8bc4914a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:27.150 7f8bc4914a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:27.337 7f8bc4914a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:27.337 7f8bc4914a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:27.728 7f8bc4914a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:27.728 7f8bc4914a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:27.986 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:28.429 7f8bc4914a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:28.430 7f8bc4914a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:28.987 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:29.493 7f8bc4914a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:29.494 7f8bc4914a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:29.988 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:30.982 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:30.983 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:30.983 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:30.983 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:30.984 7f8bc47ebb80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:30.984 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:30.984 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:30.984 7f8bc47ebb80 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:38:30.985 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:30.985 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:30.986 7f8bc47ebb80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:30.986 7f8bc47eb5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:38:30.987 7f8bc47eb5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:38:30.989 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:31.427 7f8bc4914a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:31.428 7f8bc4914a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:31.990 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:32.991 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:33.992 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:34.993 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:35.995 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:36.231 7f8bc4914a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:36.232 7f8bc4914a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:36.996 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:37.997 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:38.079 7f8bc47eb5e0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:38.998 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:39.999 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:41.000 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:42.001 7f8bc4965f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:42.058 7f8bc4914a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:38:42.058 7f8bc4914a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:38:42.058 7f8bc4914a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:42.097 7f8bc49657c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:38:42.097 7f8bc9cf8f00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:38:42.097 7f8bc9cf8f00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:38:42.097 7f8bc49149a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:38:42.097 7f8bc49149a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:38:42.104 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:38:42.104 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:38:42.105 7f8bc9cf8f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:38:50.650 7fba4a917f00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:38:50.650 7fba4a917f00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:38:50.650 7fba4a917f00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:38:50.654 7fba4a917f00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:38:50.654 7fba45e7c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:38:50.654 7fba45e7c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:38:50.654 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:38:50.655 7fba45e7c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:50.655 7fba45e7c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:38:50.655 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:38:50.655 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.656 7fba45e7c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:38:50.656 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.656 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.656 7fba45e7c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353530655545 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:38:50.656 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.656 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.656 7fba45e7c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:50.657 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.657 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.657 7fba45e7c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:50.657 7fba45e7c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.657 7fba45e7ce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:50.657 7fba45e7ce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:38:50.659 7fba45e7cc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:38:50.659 7fba45e2c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.660 7fba45e2c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:50.660 7fba45e2c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.660 7fba45e2c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.660 7fba45e2c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:50.660 7fba45e2c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.714 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:38:50.722 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:38:50.723 7fba45dcf0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.723 7fba45dcf0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:50.723 7fba45dcf0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.723 7fba45dcf0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.723 7fba45dcf0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:50.723 7fba45dcf0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.724 7fba45e2cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:50.725 7fba45e2cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:50.725 7fba45e2c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.725 7fba45e2c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:50.725 7fba45e2c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.727 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:38:50.727 7fba45e2c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.727 7fba45e2c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:50.727 7fba45e2c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.739 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:50.739 7fba45dcfc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.740 7fba45dcfc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:50.740 7fba45dcfc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.742 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:50.742 7fba45dcfc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.742 7fba45dcfc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:50.742 7fba45dcfc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.743 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:38:50.743 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:38:50.743 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:38:50.743 7fba45dcff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:50.744 7fba45d00220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.744 7fba45d00220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:50.744 7fba45d00220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.744 7fba45d00220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.744 7fba45d00220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:50.744 7fba45d00220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.747 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:38:50.750 7fba45dcff40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:50.750 7fba45d00ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.750 7fba45d00ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:50.750 7fba45d00ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.764 7fba45e7c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:38:50.766 7fba45d00e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.766 7fba45d00e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:50.766 7fba45d00e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:50.793 7fba45e2cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:50.793 7fba45e2cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:50.937 7fba45e2cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:50.937 7fba45e2cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:51.243 7fba45e2cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:51.243 7fba45e2cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:51.659 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:51.861 7fba45e2cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:51.862 7fba45e2cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:52.649 7fba45e2cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:52.650 7fba45e2cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:52.660 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:53.660 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:53.704 7fba45e2cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:53.704 7fba45e2cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:54.655 7fba45d00e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:54.656 7fba45d00e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:54.656 7fba45d00e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:54.657 7fba45d00e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:54.657 7fba45d00e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:54.657 7fba45d00e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:54.657 7fba45d00e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:54.657 7fba45d00e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:38:54.659 7fba45d00e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:54.659 7fba45d00e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:54.659 7fba45d00e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:54.659 7fba45d00720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:38:54.660 7fba45d00720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:38:54.662 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:55.663 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:56.060 7fba45e2cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:38:56.061 7fba45e2cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:56.677 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:57.678 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:58.679 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:59.680 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:00.682 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:01.683 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:02.373 7fba45e2cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:02.374 7fba45e2cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:02.685 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:03.686 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:04.687 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:05.342 7fba45e2c9a0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:39:05.688 7fba45e7cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:05.689 7fba45e2cc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:39:05.689 7fba45e2cc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:39:05.690 7fba45e2cc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:05.864 7fba45e7c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:39:05.864 7fba4a917f00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:39:05.864 7fba4a917f00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:39:05.864 7fba45e2cb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:39:05.864 7fba45e2cb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:39:05.872 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:39:05.873 7fba4a917f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:39:14.556 7fd456b0d800 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:39:14.557 7fd456b0d800 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:39:14.557 7fd456b0d800 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:39:14.562 7fd456b0d800 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:39:14.562 7fd451aba2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:39:14.563 7fd451aba2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:39:14.563 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:39:14.563 7fd451aba7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:39:14.564 7fd451aba7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:39:14.564 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:39:14.565 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.565 7fd451aba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:39:14.565 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.565 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.565 7fd451aba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353554564420 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:39:14.565 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.566 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.566 7fd451aba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:14.566 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.566 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.566 7fd451aba7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:14.567 7fd451aba7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.567 7fd451abae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:14.567 7fd451abae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:39:14.569 7fd451abac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:39:14.569 7fd451a6a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.569 7fd451a6a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:14.569 7fd451a6a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.569 7fd451a6a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.570 7fd451a6a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:14.570 7fd451a6a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.633 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:39:14.641 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:39:14.642 7fd4519cc040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.642 7fd4519cc040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:14.643 7fd4519cc040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.643 7fd4519cc040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.643 7fd4519cc040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:14.643 7fd4519cc040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.643 7fd451a6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:14.644 7fd451a6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:14.645 7fd451a6a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.645 7fd451a6a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:14.645 7fd451a6a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.646 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:39:14.646 7fd451a6a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.646 7fd451a6a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:14.646 7fd451a6a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.656 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:14.667 7fd4519ccae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.668 7fd4519ccae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:14.668 7fd4519ccae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.671 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:14.671 7fd4519ccae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.671 7fd4519ccae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:14.671 7fd4519ccae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.671 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:39:14.672 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:39:14.672 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:39:14.672 7fd4519cce00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:14.672 7fd4519350e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.673 7fd4519350e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:14.673 7fd4519350e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.673 7fd4519350e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.673 7fd4519350e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:14.673 7fd4519350e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.675 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:39:14.679 7fd4519cce00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:14.679 7fd451935ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.680 7fd451935ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:14.680 7fd451935ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.694 7fd451aba7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:39:14.706 7fd451935e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.707 7fd451935e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:14.707 7fd451935e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:14.749 7fd451a6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:14.749 7fd451a6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:14.948 7fd451a6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:14.949 7fd451a6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:15.318 7fd451a6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:15.319 7fd451a6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:15.569 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:15.969 7fd451a6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:15.969 7fd451a6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:16.570 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:17.464 7fd451a6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:17.465 7fd451a6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:17.571 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:18.564 7fd451935e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:18.565 7fd451935e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:18.566 7fd451935e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:18.566 7fd451935e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:18.566 7fd451935e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:18.566 7fd451935e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:18.566 7fd451935e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:18.566 7fd451935e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:39:18.568 7fd451935e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:18.568 7fd451935e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:18.568 7fd451935e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:18.568 7fd4519355e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:39:18.569 7fd4519355e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:39:18.573 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:19.574 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:20.575 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:20.935 7fd451a6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:20.936 7fd451a6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:21.577 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:22.578 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:23.579 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:24.580 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:25.581 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:26.583 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:26.952 7fd451a6aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:26.953 7fd451a6aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:27.584 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:28.585 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:29.587 7fd451abaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:29.664 7fd451a6aa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:39:29.664 7fd451a6aa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:39:29.665 7fd451a6aa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:29.777 7fd451aba7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:39:29.777 7fd456b0d800 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:39:29.778 7fd456b0d800 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:39:29.778 7fd451a6a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:39:29.778 7fd451a6a9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:39:29.784 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:39:29.785 7fd456b0d800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:39:38.265 7ff106c55f40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:39:38.265 7ff106c55f40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:39:38.265 7ff106c55f40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:39:38.268 7ff106c55f40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:39:38.269 7ff101ffe2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:39:38.269 7ff101ffe2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:39:38.269 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:39:38.269 7ff101ffe7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:39:38.270 7ff101ffe7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:39:38.270 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:39:38.270 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.270 7ff101ffe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:39:38.270 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.270 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.271 7ff101ffe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353578270193 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:39:38.271 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.271 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.271 7ff101ffe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:38.272 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.272 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.272 7ff101ffe7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:38.272 7ff101ffe7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.272 7ff101ffee00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:38.272 7ff101ffee00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:39:38.274 7ff101ffec20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:39:38.274 7ff101fae540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.274 7ff101fae540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:38.274 7ff101fae540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.274 7ff101fae540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.275 7ff101fae540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:38.275 7ff101fae540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.328 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:39:38.336 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:39:38.337 7ff101f0f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.337 7ff101f0f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:38.337 7ff101f0f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.337 7ff101f0f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.337 7ff101f0f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:38.338 7ff101f0f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.338 7ff101faea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:38.339 7ff101faea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:38.339 7ff101fae7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.340 7ff101fae7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:38.340 7ff101fae7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.340 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:39:38.341 7ff101fae7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.341 7ff101fae7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:38.341 7ff101fae7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.350 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:38.351 7ff101f0fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.351 7ff101f0fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:38.351 7ff101f0fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.353 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:38.353 7ff101f0fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.354 7ff101f0fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:38.354 7ff101f0fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.354 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:39:38.354 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:39:38.354 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:39:38.354 7ff101f0fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:38.355 7ff101e820e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.355 7ff101e820e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:38.355 7ff101e820e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.355 7ff101e820e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.355 7ff101e820e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:38.355 7ff101e820e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.358 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:39:38.361 7ff101f0fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:38.361 7ff101e829a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.361 7ff101e829a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:38.361 7ff101e829a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.376 7ff101ffe7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:39:38.377 7ff101e82cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.377 7ff101e82cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:38.377 7ff101e82cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:38.438 7ff101faea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:38.438 7ff101faea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:38.651 7ff101faea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:38.652 7ff101faea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:39.181 7ff101faea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:39.181 7ff101faea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:39.274 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:40.054 7ff101faea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:40.055 7ff101faea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:40.275 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:41.277 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:41.717 7ff101faea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:41.718 7ff101faea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:42.269 7ff1046d20e0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:42.272 7ff1046d20e0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:42.272 7ff1046d20e0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:42.272 7ff1046d20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:42.273 7ff1046d20e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:42.273 7ff1046d20e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:42.273 7ff1046d20e0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:42.273 7ff1046d20e0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:39:42.273 7ff1046d20e0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:42.273 7ff1046d20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:42.273 7ff1046d20e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:42.274 7ff101e82720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:39:42.275 7ff101e82720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:39:42.278 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:43.280 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:44.281 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:45.081 7ff101faea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:45.082 7ff101faea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:45.282 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:46.284 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:47.285 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:48.287 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:49.289 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:50.290 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:51.292 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:52.294 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:52.448 7ff101faea40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:39:52.449 7ff101faea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:53.295 7ff101ffef40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:53.353 7ff101faea40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:39:53.353 7ff101faea40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:39:53.353 7ff101faea40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:53.461 7ff101ffe7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:39:53.461 7ff106c55f40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:39:53.461 7ff106c55f40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:39:53.461 7ff101fae9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:39:53.461 7ff101fae9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:39:53.468 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:39:53.468 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:39:53.468 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:39:53.468 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:39:53.469 7ff106c55f40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:40:01.721 7f8114276980 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:40:01.721 7f8114276980 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:40:01.721 7f8114276980 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:40:01.724 7f8114276980 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:40:01.725 7f810f22c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:40:01.725 7f810f22c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:40:01.725 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:40:01.725 7f810f22c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:01.726 7f810f22c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:40:01.726 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:40:01.726 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.726 7f810f22c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:40:01.726 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.726 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.726 7f810f22c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353601725997 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:40:01.726 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.727 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.727 7f810f22c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:01.727 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.727 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.727 7f810f22c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:01.727 7f810f22c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.727 7f810f22ce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:01.728 7f810f22ce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:40:01.729 7f810f22cc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:40:01.729 7f810f1dc540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.729 7f810f1dc540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:01.729 7f810f1dc540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.729 7f810f1dc540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.729 7f810f1dc540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:01.729 7f810f1dc540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.782 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:40:01.791 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:40:01.792 7f810f17e040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.792 7f810f17e040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:01.792 7f810f17e040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.793 7f810f17e040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.793 7f810f17e040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:01.793 7f810f17e040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.793 7f810f1dca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:01.795 7f810f1dca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.796 7f810f1dc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.796 7f810f1dc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:01.796 7f810f1dc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.798 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:40:01.798 7f810f1dc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.798 7f810f1dc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:01.798 7f810f1dc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.809 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:01.809 7f810f17eae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.809 7f810f17eae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:01.809 7f810f17eae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.812 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:01.812 7f810f17eae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.812 7f810f17eae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:01.812 7f810f17eae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.813 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:40:01.813 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:40:01.813 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:40:01.813 7f810f17ee00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:01.814 7f810f0b10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.814 7f810f0b10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:01.814 7f810f0b10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.814 7f810f0b10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.814 7f810f0b10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:01.814 7f810f0b10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.817 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:40:01.820 7f810f17ee00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:01.820 7f810f0b19a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.820 7f810f0b19a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:01.820 7f810f0b19a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.835 7f810f22c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:40:01.836 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.836 7f810f0b1cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:01.836 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:01.893 7f810f1dca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:01.893 7f810f1dca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:02.143 7f810f1dca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:02.144 7f810f1dca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:02.729 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:02.791 7f810f1dca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:02.791 7f810f1dca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:03.730 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:04.070 7f810f1dca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:04.071 7f810f1dca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:04.732 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:05.726 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:05.727 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:05.727 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:05.727 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:05.728 7f810f0b1cc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:05.728 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:05.728 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:05.728 7f810f0b1cc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:40:05.730 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:05.730 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:05.730 7f810f0b1cc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:05.730 7f810f0b15e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:40:05.731 7f810f0b15e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:40:05.733 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:06.698 7f810f1dca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:06.698 7f810f1dca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:06.734 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:07.735 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:08.736 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:09.736 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:10.737 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:11.739 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:12.740 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:12.867 7f810f1dca40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:12.868 7f810f1dca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:13.741 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:14.742 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:15.744 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:16.745 7f810f22cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:16.788 7f810f1dca40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:40:16.788 7f810f1dca40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:40:16.789 7f810f1dca40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:16.904 7f810f22c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:40:16.904 7f8114276980 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:40:16.905 7f8114276980 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:40:16.905 7f810f1dc9a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:40:16.905 7f810f1dc9a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:40:16.912 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:40:16.912 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:40:16.912 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:40:16.912 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:40:16.912 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:40:16.912 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:40:16.913 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:40:16.913 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:40:16.913 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:40:16.913 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:40:16.913 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:40:16.913 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:40:16.913 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:40:16.913 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:40:16.913 7f8114276980 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:40:25.275 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:40:25.275 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:40:25.275 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:40:25.278 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:40:25.279 7f07386392c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:40:25.279 7f07386392c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:40:25.279 7f07386397c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:40:25.280 7f07386397c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:25.280 7f07386397c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:40:25.280 7f07386397c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:40:25.280 7f07386397c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.280 7f07386397c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:40:25.280 7f07386397c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.280 7f07386397c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.281 7f07386397c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353625280490 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:40:25.281 7f07386397c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.281 7f07386397c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.281 7f07386397c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:25.281 7f07386397c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.281 7f07386397c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.281 7f07386397c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.282 7f07386397c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.282 7f0738639e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:25.282 7f0738639e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:40:25.283 7f0738639c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:40:25.283 7f07385e9540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.284 7f07385e9540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:25.284 7f07385e9540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.284 7f07385e9540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.284 7f07385e9540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.284 7f07385e9540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.337 7f07386397c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:40:25.344 7f07386397c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:40:25.345 7f073858c040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.345 7f073858c040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:25.345 7f073858c040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.345 7f073858c040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.346 7f073858c040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:25.346 7f073858c040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.346 7f07385e9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:25.347 7f07385e97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.347 7f07385e97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:25.347 7f07385e97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.348 7f07385e9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.362 7f07386397c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:40:25.362 7f07385e97c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.362 7f07385e97c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.362 7f07385e97c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.372 7f07386397c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:25.373 7f073858cae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.373 7f073858cae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:25.373 7f073858cae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.375 7f07386397c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:25.376 7f073858cae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.376 7f073858cae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.376 7f073858cae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.376 7f07386397c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:40:25.376 7f07386397c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:40:25.377 7f07386397c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:40:25.377 7f073858ce00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:25.377 7f07384bd0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.377 7f07384bd0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:25.377 7f07384bd0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.377 7f07384bd0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.378 7f07384bd0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:25.378 7f07384bd0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.380 7f07386397c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:40:25.383 7f073858ce00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:25.384 7f07384bd9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.384 7f07384bd9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:25.384 7f07384bd9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.399 7f07386397c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:40:25.401 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.402 7f07384bdcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:25.402 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.434 7f07385e9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:25.457 7f07385e9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.629 7f07385e9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:25.630 7f07385e9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:26.081 7f07385e9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:26.082 7f07385e9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:26.283 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:26.754 7f07385e9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:26.755 7f07385e9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:27.284 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:27.874 7f07384bdcc0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:28.285 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:28.513 7f07385e9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:28.514 7f07385e9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:29.280 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:29.282 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:29.282 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:29.282 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:29.282 7f07384bdcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:29.282 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:29.282 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:29.282 7f07384bdcc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:40:29.282 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:29.283 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:29.283 7f07384bdcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:29.283 7f07384bd720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:40:29.284 7f07384bd720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:40:29.286 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:30.327 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:31.328 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:31.477 7f07385e9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:31.478 7f07385e9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:32.349 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:33.350 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:34.351 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:35.353 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:36.355 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:37.355 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:38.058 7f07385e9a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:38.059 7f07385e9a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:38.356 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:39.358 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:40.359 7f0738639f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:40.370 7f07385e9a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:40:40.370 7f07385e9a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:40:40.371 7f07385e9a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:40.448 7f07386397c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:40:40.449 7f073d2a2dc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:40:40.449 7f073d2a2dc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:40:40.449 7f07385e99a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:40:40.449 7f07385e99a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:40:40.456 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:40:40.456 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:40:40.456 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:40:40.456 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:40:40.456 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:40:40.456 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:40:40.456 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:40:40.457 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:40:40.457 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:40:40.457 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:40:40.457 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:40:40.457 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:40:40.457 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:40:40.457 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:40:40.457 7f073d2a2dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:40:48.937 7f3bfd389c00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:40:48.937 7f3bfd389c00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:40:48.937 7f3bfd389c00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:40:48.940 7f3bfd389c00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:40:48.941 7f3bf85bf2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:40:48.941 7f3bf85bf2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:40:48.941 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:40:48.942 7f3bf85bf7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:48.942 7f3bf85bf7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:40:48.942 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:40:48.942 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.942 7f3bf85bf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:40:48.942 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.942 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.942 7f3bf85bf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353648942276 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:40:48.943 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.943 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.943 7f3bf85bf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:48.943 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.943 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.943 7f3bf85bf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:48.944 7f3bf85bf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.944 7f3bf85bfe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:48.944 7f3bf85bfe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:40:48.945 7f3bf85bfc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:40:48.946 7f3bf856f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.946 7f3bf856f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:48.946 7f3bf856f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.946 7f3bf856f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:48.946 7f3bf856f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:48.947 7f3bf856f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.002 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:40:49.010 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:40:49.011 7f3bfa6d60e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.011 7f3bfa6d60e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:49.011 7f3bfa6d60e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.011 7f3bfa6d60e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.011 7f3bfa6d60e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:49.011 7f3bfa6d60e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.012 7f3bf856fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:49.013 7f3bf856fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:49.014 7f3bf856f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.014 7f3bf856f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:49.014 7f3bf856f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.016 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:40:49.016 7f3bf856f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.016 7f3bf856f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:49.016 7f3bf856f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.027 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:49.027 7f3bfa6d6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.027 7f3bfa6d6c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:49.028 7f3bfa6d6c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.030 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:49.030 7f3bfa6d6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.030 7f3bfa6d6c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:49.030 7f3bfa6d6c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.031 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:40:49.031 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:40:49.031 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:40:49.031 7f3bfa6d6f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:49.032 7f3bf9b96220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.032 7f3bf9b96220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:49.032 7f3bf9b96220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.032 7f3bf9b96220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.032 7f3bf9b96220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:49.032 7f3bf9b96220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.035 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:40:49.038 7f3bfa6d6f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:49.038 7f3bf9b96ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.038 7f3bf9b96ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:49.038 7f3bf9b96ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.054 7f3bf85bf7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:40:49.055 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.056 7f3bf9b96e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:49.056 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.137 7f3bf856fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:49.138 7f3bf856fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:49.338 7f3bf856fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:49.339 7f3bf856fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:49.785 7f3bf856fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:49.786 7f3bf856fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:49.945 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:50.353 7f3bf856fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:50.354 7f3bf856fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.946 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:51.350 7f3bf856fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:51.351 7f3bf856fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:51.947 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:52.942 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:52.944 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:52.944 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:52.944 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:52.944 7f3bf9b96e00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:52.944 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:52.944 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:52.944 7f3bf9b96e00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:40:52.946 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:52.946 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:52.946 7f3bf9b96e00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:52.947 7f3bf9b96720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:40:52.948 7f3bf9b96720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:40:52.948 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:53.155 7f3bf856fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:53.156 7f3bf856fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:53.950 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:54.951 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:55.634 7f3bf856fb80 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:55.826 7f3bf856fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:55.826 7f3bf856fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:55.952 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:56.953 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:57.954 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:58.956 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:59.644 7f3bf856fc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:40:59.645 7f3bf856fc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:59.957 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:00.958 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:01.959 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:02.960 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:03.961 7f3bf85bff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:03.963 7f3bf856fc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:41:03.963 7f3bf856fc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:41:03.964 7f3bf856fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:04.084 7f3bf85bf7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:41:04.084 7f3bfd389c00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:41:04.084 7f3bfd389c00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:41:04.084 7f3bf856fa40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:41:04.084 7f3bf856fa40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:41:04.092 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:41:04.093 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:41:04.093 7f3bfd389c00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:41:12.599 7f4701a94bc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:41:12.600 7f4701a94bc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:41:12.600 7f4701a94bc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:41:12.604 7f4701a94bc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:41:12.604 7f46fcccc2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:41:12.604 7f46fcccc2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:41:12.604 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:41:12.605 7f46fcccc7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:41:12.605 7f46fcccc7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:41:12.606 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:41:12.606 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.606 7f46fcccc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:41:12.606 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.606 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.606 7f46fcccc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353672605783 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:41:12.606 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.607 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.607 7f46fcccc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:12.607 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.608 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.608 7f46fcccc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:12.608 7f46fcccc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.608 7f46fccccf40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:12.608 7f46fccccf40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:41:12.609 7f46fcccce00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:41:12.610 7f46fcc7b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.610 7f46fcc7b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:12.610 7f46fcc7b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.610 7f46fcc7b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.610 7f46fcc7b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:12.610 7f46fcc7b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.664 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:41:12.672 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:41:12.673 7f46fcc1a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.673 7f46fcc1a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:12.673 7f46fcc1a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.673 7f46fcc1a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.673 7f46fcc1a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:12.673 7f46fcc1a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.673 7f46fcc7bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:12.674 7f46fcc7bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:12.675 7f46fcc7b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.675 7f46fcc7b9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:12.675 7f46fcc7b9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.676 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:41:12.676 7f46fcc7b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.676 7f46fcc7b9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:12.676 7f46fcc7b9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.686 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:12.686 7f46fcc1ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.687 7f46fcc1ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:12.687 7f46fcc1ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.689 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:12.689 7f46fcc1ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.689 7f46fcc1ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:12.689 7f46fcc1ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.690 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:41:12.690 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:41:12.690 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:41:12.690 7f46fcc1af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:12.691 7f46fcb50220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.691 7f46fcb50220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:12.691 7f46fcb50220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.691 7f46fcb50220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.691 7f46fcb50220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:12.691 7f46fcb50220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.693 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:41:12.702 7f46fcc1af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:12.702 7f46fcb50d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.703 7f46fcb50d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:12.703 7f46fcb50d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.713 7f46fcccc7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:41:12.714 7f46fcb50f40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.714 7f46fcb50f40 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:12.714 7f46fcb50f40 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:12.747 7f46fcc7bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:12.747 7f46fcc7bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:12.882 7f46fcc7bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:12.882 7f46fcc7bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:12.903 7f46fcb50f40 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:41:13.145 7f46fcc7bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:13.145 7f46fcc7bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:13.553 7f46fcc7bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:13.553 7f46fcc7bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:13.610 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:14.611 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:14.681 7f46fcc7bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:14.682 7f46fcc7bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:15.613 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:16.606 7f46fcc1aea0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:16.607 7f46fcc1aea0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:16.607 7f46fcc1aea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:16.607 7f46fcc1aea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:16.607 7f46fcc1aea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:16.607 7f46fcc1aea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:16.608 7f46fcc1aea0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:16.608 7f46fcc1aea0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:41:16.608 7f46fcc1aea0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:16.608 7f46fcc1aea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:16.608 7f46fcc1aea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:16.608 7f46fcb50900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:41:16.609 7f46fcb50900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:41:16.614 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:17.614 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:17.775 7f46fcc7bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:17.775 7f46fcc7bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:18.616 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:19.617 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:20.618 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:21.620 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:22.578 7f46fcc7bc20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:22.579 7f46fcc7bc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:22.621 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:23.622 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:24.623 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:25.625 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:26.626 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:27.627 7f46fcc7b2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:27.700 7f46fcc7bc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:41:27.700 7f46fcc7bc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:41:27.701 7f46fcc7bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:27.806 7f46fcccc7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:41:27.807 7f4701a94bc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:41:27.807 7f4701a94bc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:41:27.807 7f46fcc7bb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:41:27.807 7f46fcc7bb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:41:27.814 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:41:27.814 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:41:27.814 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:41:27.814 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:41:27.814 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:41:27.814 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:41:27.815 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:41:27.815 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:41:27.815 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:41:27.815 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:41:27.815 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:41:27.815 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:41:27.815 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:41:27.815 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:41:27.815 7f4701a94bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:41:36.415 7fd9733060c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:41:36.415 7fd9733060c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:41:36.415 7fd9733060c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:41:36.418 7fd9733060c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:41:36.418 7fd96e8772c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:41:36.418 7fd96e8772c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:41:36.419 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:41:36.419 7fd96e8777c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:41:36.419 7fd96e8777c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:41:36.419 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:41:36.419 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.420 7fd96e8777c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:41:36.420 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.420 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.420 7fd96e8777c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353696419653 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:41:36.420 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.420 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.420 7fd96e8777c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:36.420 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.421 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.421 7fd96e8777c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:36.421 7fd96e8777c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.421 7fd96e877e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:36.421 7fd96e877e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:41:36.422 7fd96e877c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:41:36.422 7fd96e826540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.423 7fd96e826540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:36.423 7fd96e826540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.423 7fd96e826540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.423 7fd96e826540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:36.423 7fd96e826540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.474 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:41:36.481 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:41:36.482 7fd96e7c8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.482 7fd96e7c8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:36.482 7fd96e7c8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.482 7fd96e7c8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.482 7fd96e7c8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:36.482 7fd96e7c8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.483 7fd96e826a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:36.484 7fd96e826a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:36.484 7fd96e8267c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.484 7fd96e8267c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:36.484 7fd96e8267c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.485 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:41:36.486 7fd96e8267c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.486 7fd96e8267c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:36.486 7fd96e8267c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.496 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:36.497 7fd96e7c8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.497 7fd96e7c8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:36.497 7fd96e7c8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.499 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:36.499 7fd96e7c8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.500 7fd96e7c8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:36.500 7fd96e7c8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.500 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:41:36.500 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:41:36.500 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:41:36.501 7fd96e7c8e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:36.501 7fd96e6fc0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.501 7fd96e6fc0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:36.501 7fd96e6fc0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.501 7fd96e6fc0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.501 7fd96e6fc0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:36.501 7fd96e6fc0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.504 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:41:36.506 7fd96e7c8e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:36.507 7fd96e6fc9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.507 7fd96e6fc9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:36.507 7fd96e6fc9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.523 7fd96e8777c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:41:36.525 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.525 7fd96e6fccc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:36.525 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:36.597 7fd96e826a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:36.598 7fd96e826a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:36.899 7fd96e826a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:36.899 7fd96e826a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:37.422 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:37.613 7fd96e826a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:37.613 7fd96e826a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:38.423 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:38.608 7fd96e826a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:38.609 7fd96e826a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:39.425 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:39.842 7fd96e826a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:39.843 7fd96e826a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.420 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:40.421 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:40.421 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:40.421 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:40.421 7fd96e6fccc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:40.421 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:40.421 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:40.422 7fd96e6fccc0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:41:40.423 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:40.423 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:40.424 7fd96e6fccc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:40.424 7fd96e6fc5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:41:40.425 7fd96e6fc5e0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:41:40.426 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:41.427 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:42.429 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:43.254 7fd96e826a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:43.255 7fd96e826a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:43.430 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:44.431 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:45.432 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:46.434 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:47.435 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:48.436 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:49.437 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:50.439 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:51.440 7fd96e877f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:51.492 7fd96e826a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:41:51.492 7fd96e826a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:41:51.493 7fd96e826a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:51.571 7fd96e8777c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:41:51.571 7fd9733060c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:41:51.571 7fd9733060c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:41:51.571 7fd96e8269a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:41:51.571 7fd96e8269a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:41:51.580 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:41:51.581 7fd9733060c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:41:59.845 7f6323aa9700 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:41:59.845 7f6323aa9700 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:41:59.845 7f6323aa9700 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:41:59.848 7f6323aa9700 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:41:59.849 7f631ea632c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:41:59.849 7f631ea632c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:41:59.849 7f631ea637c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:41:59.849 7f631ea637c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:41:59.849 7f631ea637c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:41:59.850 7f631ea637c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:41:59.850 7f631ea637c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.850 7f631ea637c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:41:59.850 7f631ea637c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.850 7f631ea637c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.850 7f631ea637c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353719849911 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:41:59.850 7f631ea637c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.851 7f631ea637c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.851 7f631ea637c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:59.851 7f631ea637c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.851 7f631ea637c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.851 7f631ea637c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:59.851 7f631ea637c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.852 7f631ea63f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:59.852 7f631ea63f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:41:59.853 7f631ea63c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:41:59.853 7f631ea137c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.854 7f631ea137c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:59.854 7f631ea137c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.854 7f631ea137c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.854 7f631ea137c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:59.854 7f631ea137c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.906 7f631ea637c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:41:59.913 7f631ea637c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:41:59.914 7f631e9fc0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.914 7f631e9fc0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:59.914 7f631e9fc0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.914 7f631e9fc0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.914 7f631e9fc0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:59.914 7f631e9fc0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.915 7f631ea13c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:41:59.915 7f631ea13c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:59.916 7f631ea139a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.917 7f631ea139a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:59.917 7f631ea139a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.918 7f631ea637c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:41:59.918 7f631ea139a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.919 7f631ea139a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:59.919 7f631ea139a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.929 7f631ea637c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:59.930 7f631e9fcc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.930 7f631e9fcc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:59.930 7f631e9fcc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.933 7f631ea637c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:59.933 7f631e9fcc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.933 7f631e9fcc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:59.934 7f631e9fcc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.934 7f631ea637c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:41:59.934 7f631ea637c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:41:59.934 7f631ea637c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:41:59.935 7f631e9fcf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:59.935 7f632003d220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.935 7f632003d220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:59.935 7f632003d220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.936 7f632003d220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.936 7f632003d220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:59.936 7f632003d220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.939 7f631ea637c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:41:59.942 7f631e9fcf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:59.943 7f632003dae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.943 7f632003dae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:59.943 7f632003dae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.957 7f631ea637c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:41:59.958 7f632003de00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:59.958 7f632003de00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:59.958 7f632003de00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:00.018 7f631ea13c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:00.019 7f631ea13c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:00.261 7f631ea13c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:00.262 7f631ea13c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:00.561 7f631ea13c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:00.562 7f631ea13c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:00.853 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:01.295 7f631ea13c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:01.296 7f631ea13c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:01.854 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:02.855 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:03.161 7f631ea13c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:03.162 7f631ea13c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:03.850 7f632003de00 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:03.851 7f632003de00 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:03.851 7f632003de00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:03.851 7f632003de00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:03.851 7f632003de00 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.185:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:03.851 7f632003de00 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:03.852 7f632003de00 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:03.852 7f632003de00 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:42:03.853 7f632003de00 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:03.853 7f632003de00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:03.854 7f632003de00 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:03.854 7f632003d720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:42:03.855 7f632003d720 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:42:03.857 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:04.858 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:05.859 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:06.327 7f631ea13c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:06.328 7f631ea13c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:06.860 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:07.861 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:08.862 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:09.863 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:10.865 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:11.028 7f631ea63ea0 [contrail-alarm-gen] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:42:11.867 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:12.867 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:13.868 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:14.076 7f631ea13c20 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:14.077 7f631ea13c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:14.870 7f631ea132c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:14.881 7f631ea13c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:42:14.881 7f631ea13c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:42:14.882 7f631ea13c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:14.967 7f631ea637c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:42:14.967 7f6323aa9700 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:42:14.968 7f6323aa9700 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:42:14.968 7f631ea13b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:42:14.968 7f631ea13b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:42:14.975 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:42:14.976 7f6323aa9700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:42:23.704 7f764513be40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:42:23.704 7f764513be40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:42:23.704 7f764513be40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:42:23.709 7f764513be40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:42:23.709 7f76406752c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:42:23.709 7f76406752c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:42:23.710 7f76406757c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:42:23.710 7f76406757c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:42:23.711 7f76406757c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:42:23.711 7f76406757c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:42:23.711 7f76406757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.711 7f76406757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:42:23.711 7f76406757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.711 7f76406757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.711 7f76406757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353743710998 collector_name = collector_ip = collector_list = [ 10.0.0.185:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:42:23.711 7f76406757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.712 7f76406757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.712 7f76406757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:23.712 7f76406757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.713 7f76406757c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.713 7f76406757c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:23.713 7f76406757c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.713 7f7640675e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:23.713 7f7640675e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:42:23.715 7f7640675c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:42:23.715 7f7640625540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.715 7f7640625540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6276-1.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:23.716 7f7640625540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.716 7f7640625540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.716 7f7640625540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:23.716 7f7640625540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.778 7f76406757c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.185:5673//'] 09/14/2026 02:42:23.785 7f76406757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:42:23.786 7f76405c6040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.787 7f76405c6040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:23.787 7f76405c6040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.787 7f76405c6040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.787 7f76405c6040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.185:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:23.787 7f76405c6040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.787 7f7640625a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:23.788 7f7640625a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:23.789 7f76406257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.789 7f76406257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:23.789 7f76406257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.790 7f76406757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:42:23.791 7f76406257c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.791 7f76406257c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:23.791 7f76406257c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.801 7f76406757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:23.801 7f76405c6ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.801 7f76405c6ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:23.801 7f76405c6ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.804 7f76406757c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:23.804 7f76405c6ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.805 7f76405c6ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:23.805 7f76405c6ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.805 7f76406757c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:42:23.805 7f76406757c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:42:23.805 7f76406757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:42:23.805 7f76405c6e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:23.806 7f76404fb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.806 7f76404fb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:23.806 7f76404fb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.806 7f76404fb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.806 7f76404fb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:23.806 7f76404fb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.809 7f76406757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:42:23.812 7f76405c6e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:23.812 7f76404fb9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.812 7f76404fb9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:23.812 7f76404fb9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.827 7f76406757c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:42:23.829 7f76404fbcc0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.829 7f76404fbcc0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6276-1. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.185:6379 server_addrs = [ 10.0.0.185:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.185:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.185:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.185:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:23.829 7f76404fbcc0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:23.899 7f7640625a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:23.899 7f7640625a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:24.098 7f7640625a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:24.099 7f7640625a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:24.623 7f7640625a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:24.623 7f7640625a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:24.715 7f7640675f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:25.716 7f7640675f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:25.718 7f7640625a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:25.718 7f7640625a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:26.717 7f7640675f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:27.036 7f7640625a40 [kazoo.client] [INFO]: Connecting to 10.0.0.185(10.0.0.185):2181, use_ssl: False 09/14/2026 02:42:27.036 7f7640625a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused