09/14/2026 02:30:42.563 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:30:42.563 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:30:42.563 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:30:42.572 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:30:42.573 7f7c7dcca2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:30:42.573 7f7c7dcca2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:30:42.573 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:30:42.574 7f7c7dcca7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:30:42.574 7f7c7dcca7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:30:42.574 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:30:42.574 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.575 7f7c7dcca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:30:42.575 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.575 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.575 7f7c7dcca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353042574545 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:30:42.575 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.576 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:42.577 7f7c7dcca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.578 7f7c7dc7a2c0 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:42.578 7f7c7dc7a2c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:30:42.579 7f7c7dccae00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:30:42.580 7f7c7dc7a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.580 7f7c7dc7a9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:30:42.580 7f7c7dc7a9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.580 7f7c7dc7a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.580 7f7c7dc7a9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:42.580 7f7c7dc7a9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.632 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:30:42.639 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:30:42.640 7f7c7dc5a040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.640 7f7c7dc5a040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:42.641 7f7c7dc5a040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.641 7f7c7dc5a040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.641 7f7c7dc5a040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:42.641 7f7c7dc5a040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.642 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:42.643 7f7c7dc7ab80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.643 7f7c7dc7ab80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:42.643 7f7c7dc7ab80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.644 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:42.644 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:42.645 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:30:42.645 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:42.645 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:42.646 7f7c7dc7ab80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.646 7f7c7dc7ab80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:42.646 7f7c7dc7ab80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.646 7f7c7dc7ab80 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.647 7f7c7dc7ab80 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:42.647 7f7c7dc7ab80 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.647 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:42.654 7f7c7dc5ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.654 7f7c7dc5ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:42.654 7f7c7dc5ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.685 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:42.686 7f7c7dc5aae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.686 7f7c7dc5aae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:30:42.686 7f7c7dc5aae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.690 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:42.690 7f7c7dc5aae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.691 7f7c7dc5aae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:30:42.691 7f7c7dc5aae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.691 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:30:42.691 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:30:42.691 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:30:42.692 7f7c7dc5ae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:30:42.692 7f7c7d5980e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.692 7f7c7d5980e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:42.692 7f7c7d5980e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.692 7f7c7d5980e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.692 7f7c7d5980e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:42.693 7f7c7d5980e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.696 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:30:42.698 7f7c7dc5ae00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:30:42.698 7f7c7d5989a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.698 7f7c7d5989a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:30:42.699 7f7c7d5989a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.738 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:42.739 7f7c7dcca7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:30:42.739 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:42.739 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:42.741 7f7c7fdd4180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.741 7f7c7fdd4180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:30:42.741 7f7c7fdd4180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:42.741 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:42.741 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:42.742 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:42.928 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:42.929 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:42.929 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:42.930 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:42.930 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:42.930 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:43.268 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:43.269 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:43.269 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:43.269 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:43.269 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:43.270 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:43.579 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:43.937 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:43.938 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:43.938 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:43.938 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:43.939 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:43.939 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:44.580 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:45.581 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:45.763 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:45.764 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.765 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:45.765 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:45.765 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:45.766 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:46.574 7f7c7fdd4680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:46.575 7f7c7fdd4680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:30:46.575 7f7c7fdd4680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:46.575 7f7c7fdd4680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:46.575 7f7c7fdd4680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:30:46.575 7f7c7fdd4680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:46.575 7f7c7fdd4680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:30:46.576 7f7c7fdd4680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:30:46.576 7f7c7fdd4680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:30:46.576 7f7c7fdd4680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:46.576 7f7c7fdd4680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:46.576 7f7c7fdd4900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:30:46.577 7f7c7fdd4900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:30:46.583 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:47.584 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:48.585 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:49.178 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:49.179 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:49.179 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:49.180 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:49.180 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:49.180 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:49.586 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:50.587 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:51.588 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:52.588 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:53.590 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:54.592 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:55.594 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:56.595 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:57.450 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:30:57.451 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:57.451 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:30:57.454 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:57.455 7f7c7dc7aea0 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:30:57.455 7f7c7dc7aea0 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:30:57.595 7f7c7dc7a400 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:30:57.656 7f7c7dc7aea0 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:30:57.656 7f7c7dc7aea0 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:30:57.657 7f7c808634a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:57.798 7f7c7dcca7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:30:57.799 7f7c82d1c9c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:30:57.799 7f7c82d1c9c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:30:57.799 7f7c7dc7aa40 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:30:57.799 7f7c7dc7aa40 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:30:57.814 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:30:57.815 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:30:57.815 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:30:57.815 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:30:57.815 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:30:57.815 7f7c82d1c9c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:31:07.130 7faa101a1e80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:31:07.130 7faa101a1e80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:31:07.130 7faa101a1e80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:31:07.134 7faa101a1e80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:31:07.135 7faa0b7122c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:31:07.135 7faa0b7122c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:31:07.135 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:31:07.136 7faa0b7127c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:31:07.136 7faa0b7127c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:31:07.137 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:31:07.137 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.137 7faa0b7127c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:31:07.137 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.137 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.137 7faa0b7127c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353067136613 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:31:07.137 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.138 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.138 7faa0b7127c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:07.139 7faa0b7127c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.140 7faa0b712e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:07.140 7faa0b712e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:31:07.141 7faa0b712c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:31:07.142 7faa0b6c2540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.142 7faa0b6c2540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:07.142 7faa0b6c2540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.142 7faa0b6c2540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.142 7faa0b6c2540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:07.142 7faa0b6c2540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.209 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:31:07.218 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:31:07.219 7faa0b614040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.219 7faa0b614040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:07.219 7faa0b614040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.219 7faa0b614040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.219 7faa0b614040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:07.219 7faa0b614040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.221 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:07.222 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.222 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:07.223 7faa0b6c27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.223 7faa0b6c27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:07.223 7faa0b6c27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.224 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.225 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:07.226 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:31:07.226 7faa0b6c27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.226 7faa0b6c27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:07.226 7faa0b6c27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.226 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.228 7faa0b614040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.228 7faa0b614040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:07.228 7faa0b614040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.238 7faa0b614d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.238 7faa0b614d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:07.238 7faa0b614d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.262 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:07.263 7faa0b614c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.263 7faa0b614c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:07.263 7faa0b614c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.266 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:07.267 7faa0b614c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.267 7faa0b614c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:07.267 7faa0b614c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.267 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:31:07.268 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:31:07.268 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:31:07.268 7faa0b614f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:07.269 7faa0b682220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.269 7faa0b682220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:07.269 7faa0b682220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.269 7faa0b682220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.269 7faa0b682220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:07.269 7faa0b682220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.273 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:31:07.275 7faa0b614f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:07.275 7faa0b6829a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.275 7faa0b6829a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:07.275 7faa0b6829a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.303 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:07.304 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.304 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:07.305 7faa0b7127c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:31:07.305 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.305 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:07.306 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.307 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.308 7faa0b5ae540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:07.308 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:07.411 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:07.412 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.412 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:07.413 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.413 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:07.413 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.690 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:07.691 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.691 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:07.691 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:07.691 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:07.692 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:08.141 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:08.363 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:08.364 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:08.364 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:08.365 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:08.365 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:08.366 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.142 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:09.700 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:09.701 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.701 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:09.701 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:09.702 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:09.702 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:10.143 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:11.136 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:11.137 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:11.138 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:11.138 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:11.138 7faa0b5ae540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:11.138 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:11.138 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:11.138 7faa0b5ae540 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:31:11.139 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:11.139 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:11.139 7faa0b5ae540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:11.139 7faa0b5aea40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:31:11.140 7faa0b5aea40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:31:11.144 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:12.144 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:13.145 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:13.389 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:13.389 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:13.389 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:13.390 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:13.390 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:13.391 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:14.146 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:15.147 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:16.149 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:17.150 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:18.151 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:19.152 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:19.197 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:19.198 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:19.198 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:19.199 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:19.199 7faa0b6c2a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:19.199 7faa0b6c2a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:20.153 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:21.154 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:22.155 7faa0b712f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:22.213 7faa0b6c2a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:31:22.213 7faa0b6c2a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:31:22.214 7faa0b6c2a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:22.395 7faa0b7127c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:31:22.396 7faa101a1e80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:31:22.396 7faa101a1e80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:31:22.396 7faa0b6c29a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:31:22.396 7faa0b6c29a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:31:22.411 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:31:22.411 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:31:22.411 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:31:22.411 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:31:22.411 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:31:22.412 7faa101a1e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:31:31.829 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:31:31.829 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:31:31.829 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:31:31.832 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:31:31.832 7f71efdf32c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:31:31.833 7f71efdf32c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:31:31.833 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:31:31.833 7f71efdf37c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:31:31.833 7f71efdf37c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:31:31.833 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:31:31.834 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.834 7f71efdf37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:31:31.834 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.834 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.834 7f71efdf37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353091833753 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:31:31.834 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:31.835 7f71efdf37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.836 7f71efdf3e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:31.836 7f71efdf3e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:31:31.837 7f71efdf3c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:31:31.837 7f71efda3540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.837 7f71efda3540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:31.837 7f71efda3540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.837 7f71efda3540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.837 7f71efda3540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:31.837 7f71efda3540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.888 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:31:31.896 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:31:31.896 7f71efd05040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.897 7f71efd05040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:31.897 7f71efd05040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.897 7f71efd05040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.897 7f71efd05040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:31.897 7f71efd05040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.898 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:31.899 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:31.899 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:31.900 7f71efda37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.900 7f71efda37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:31.900 7f71efda37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.901 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:31.901 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:31.902 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:31:31.902 7f71efda37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.903 7f71efda37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:31.903 7f71efda37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.903 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:31.903 7f71efda37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.904 7f71efda37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:31.904 7f71efda37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.905 7f71efd05040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.905 7f71efd05040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:31.905 7f71efd05040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.946 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:31.946 7f71efd05c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.947 7f71efd05c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:31.947 7f71efd05c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.950 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:31.951 7f71efd05c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.951 7f71efd05c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:31.951 7f71efd05c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.951 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:31:31.951 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:31:31.952 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:31:31.952 7f71efd05f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:31.952 7f71efc77220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.953 7f71efc77220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:31.953 7f71efc77220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.953 7f71efc77220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.953 7f71efc77220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:31.953 7f71efc77220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.956 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:31:31.959 7f71efd05f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:31.959 7f71efc779a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.960 7f71efc779a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:31.960 7f71efc779a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.985 7f71efdf37c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:31:31.986 7f71f1486180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:31.987 7f71f1486180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:31.987 7f71f1486180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:32.042 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:32.043 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:32.043 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:32.044 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:32.044 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:32.045 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:32.271 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:32.271 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:32.271 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:32.272 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:32.272 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:32.272 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:32.586 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:32.587 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:32.587 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:32.587 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:32.588 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:32.588 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:32.837 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:33.057 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:33.058 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:33.058 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:33.059 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:33.059 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:33.059 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:33.805 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:33.806 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:33.806 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:33.806 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:33.806 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:33.807 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:33.838 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:34.839 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:35.222 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:35.223 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:35.223 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:35.224 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:35.224 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:35.224 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:35.834 7f71efc77900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:35.836 7f71efc77900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:35.836 7f71efc77900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:35.836 7f71efc77900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:35.836 7f71efc77900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:35.836 7f71efc77900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:35.836 7f71efc77900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:35.836 7f71efc77900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:31:35.837 7f71efc77900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:35.838 7f71efc77900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:35.838 7f71efc77900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:35.838 7f71f1486680 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:31:35.840 7f71f1486680 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:31:35.840 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:36.842 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:37.106 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:37.107 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:37.107 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:37.107 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:37.107 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:37.108 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:37.842 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:38.843 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:39.845 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:40.264 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:40.265 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:40.265 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:40.266 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:40.266 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:40.266 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:40.861 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:41.862 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:42.863 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:43.864 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:44.340 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:44.341 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:44.341 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:44.341 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:44.342 7f71efda3a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:44.342 7f71efda3a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:44.865 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:45.867 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:46.855 7f71efda3a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:31:46.855 7f71efda3a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:31:46.856 7f71efda3a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:46.868 7f71efdf3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:47.033 7f71efdf37c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:31:47.033 7f71f4bc2f00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:31:47.033 7f71f4bc2f00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:31:47.033 7f71efda39a0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:31:47.033 7f71efda39a0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:31:47.082 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:31:47.082 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:31:47.082 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:31:47.082 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:31:47.082 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:31:47.083 7f71f4bc2f00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:31:55.954 7fbd25513d00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:31:55.954 7fbd25513d00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:31:55.954 7fbd25513d00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:31:55.958 7fbd25513d00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:31:55.959 7fbd20a962c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:31:55.959 7fbd20a962c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:31:55.959 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:31:55.959 7fbd20a967c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:31:55.960 7fbd20a967c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:31:55.960 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:31:55.960 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.960 7fbd20a967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:31:55.960 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.960 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.960 7fbd20a967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353115960102 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:31:55.960 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.961 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.961 7fbd20a967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:55.961 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.961 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.961 7fbd20a967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:55.961 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.961 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.962 7fbd20a967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:31:55.962 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.962 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.962 7fbd20a967c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:55.962 7fbd20a967c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.962 7fbd20a96e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:55.962 7fbd20a96e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:31:55.964 7fbd20a96c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:31:55.964 7fbd20a45540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.964 7fbd20a45540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:31:55.964 7fbd20a45540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.964 7fbd20a45540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.964 7fbd20a45540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:55.964 7fbd20a45540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.019 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:31:56.026 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:31:56.027 7fbd22ba7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.027 7fbd22ba7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:56.028 7fbd22ba7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.028 7fbd22ba7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.028 7fbd22ba7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:56.028 7fbd22ba7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.029 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:56.030 7fbd20a457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.030 7fbd20a457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:56.030 7fbd20a457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.031 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.031 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:56.032 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:31:56.033 7fbd20a457c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.033 7fbd20a457c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:56.033 7fbd20a457c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.033 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.033 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:56.034 7fbd22ba7040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.034 7fbd22ba7040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:56.034 7fbd22ba7040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.040 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.042 7fbd22ba7d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.042 7fbd22ba7d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:56.042 7fbd22ba7d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.068 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:56.069 7fbd22ba7c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.070 7fbd22ba7c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:31:56.070 7fbd22ba7c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.072 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:56.072 7fbd22ba7c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.073 7fbd22ba7c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:31:56.073 7fbd22ba7c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.073 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:31:56.073 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:31:56.073 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:31:56.074 7fbd22ba7f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:31:56.074 7fbd20a2e220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.074 7fbd20a2e220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:56.075 7fbd20a2e220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.075 7fbd20a2e220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.075 7fbd20a2e220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:56.075 7fbd20a2e220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.078 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:31:56.080 7fbd22ba7f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:31:56.080 7fbd20a2e9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.080 7fbd20a2e9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:31:56.080 7fbd20a2e9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.102 7fbd20a967c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:31:56.104 7fbd22048400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.104 7fbd22048400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:56.104 7fbd22048400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.164 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:56.165 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.165 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:56.166 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.166 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:56.166 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.338 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:56.339 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.339 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:56.340 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.340 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:56.340 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.571 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:56.572 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.573 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:56.573 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.573 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:56.574 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.963 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:56.968 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:56.969 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.969 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:56.970 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:56.970 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:56.970 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:57.451 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:57.451 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:57.451 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:57.452 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:57.452 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:57.452 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:57.964 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:58.502 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:58.503 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:58.503 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:58.504 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:58.504 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:58.504 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:58.965 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:31:59.853 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:31:59.854 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:59.854 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:31:59.855 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:59.855 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:31:59.855 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:31:59.960 7fbd22048400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:59.961 7fbd22048400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:59.961 7fbd22048400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:59.962 7fbd22048400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:59.962 7fbd22048400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:31:59.962 7fbd22048400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:59.962 7fbd22048400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:59.962 7fbd22048400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:31:59.962 7fbd22048400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:59.962 7fbd22048400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:59.962 7fbd22048400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:59.963 7fbd22048900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:31:59.963 7fbd22048900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:31:59.966 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:00.967 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:01.967 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:02.495 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:02.496 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:02.496 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:02.496 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:02.497 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:02.497 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:02.969 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:03.971 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:04.971 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:05.827 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:05.828 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:05.828 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:05.829 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:05.829 7fbd20a45a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:05.830 7fbd20a45a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:05.973 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:06.974 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:07.976 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:08.977 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:09.978 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:10.980 7fbd20a96f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:11.055 7fbd20a45a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:32:11.056 7fbd20a45a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:32:11.056 7fbd20a45a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:11.124 7fbd20a967c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:32:11.124 7fbd25513d00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:32:11.125 7fbd25513d00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:32:11.125 7fbd20a45680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:32:11.125 7fbd20a45680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:32:11.138 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:32:11.138 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:32:11.138 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:32:11.138 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:32:11.138 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:32:11.138 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:32:11.139 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:32:11.139 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:32:11.139 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:32:11.139 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:32:11.139 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:32:11.139 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:32:11.139 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:32:11.139 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:32:11.139 7fbd25513d00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:32:20.044 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:32:20.044 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:32:20.044 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:32:20.047 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:32:20.048 7f6afbb2f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:32:20.048 7f6afbb2f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:32:20.048 7f6afbb2f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:32:20.048 7f6afbb2f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:32:20.049 7f6afbb2f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:32:20.049 7f6afbb2f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:32:20.049 7f6afbb2f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.049 7f6afbb2f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:32:20.049 7f6afbb2f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.049 7f6afbb2f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.049 7f6afbb2f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353140049053 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:32:20.049 7f6afbb2f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.050 7f6afbb2fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.050 7f6afbb2fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:20.050 7f6afbb2fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.050 7f6afbb2fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.050 7f6afbb2fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:20.050 7f6afbb2fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.050 7f6afbb2fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.051 7f6afbb2fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:20.051 7f6afbb2fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.051 7f6afbb2fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.051 7f6afbb2fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:20.051 7f6afbb2fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.051 7f6afbb2ff40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:20.051 7f6afbb2ff40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:32:20.052 7f6afbb2fe00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:32:20.053 7f6afbadf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.053 7f6afbadf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:20.053 7f6afbadf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.053 7f6afbadf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.053 7f6afbadf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:20.053 7f6afbadf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.109 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:32:20.117 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:32:20.118 7f6afbba5220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.118 7f6afbba5220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:20.118 7f6afbba5220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.118 7f6afbba5220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.118 7f6afbba5220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:20.119 7f6afbba5220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.120 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:20.120 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.120 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:20.121 7f6afbadf9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.121 7f6afbadf9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:20.121 7f6afbadf9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.122 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.122 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:20.123 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.123 7f6afbadf9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.124 7f6afbadf9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:20.124 7f6afbadf9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.126 7f6afbadf9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.126 7f6afbadf9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:20.126 7f6afbadf9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.134 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:32:20.134 7f6afbadf9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.134 7f6afbadf9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:20.134 7f6afbadf9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.176 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:20.176 7f6afbba5d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.177 7f6afbba5d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:20.177 7f6afbba5d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.179 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:20.180 7f6afbba5d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.180 7f6afbba5d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:20.180 7f6afbba5d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.181 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:32:20.181 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:32:20.181 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:32:20.181 7f6afbba5d60 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:20.182 7f6afb9a7360 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.182 7f6afb9a7360 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:20.182 7f6afb9a7360 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.182 7f6afb9a7360 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.182 7f6afb9a7360 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:20.182 7f6afb9a7360 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.185 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:32:20.188 7f6afbba5d60 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:20.188 7f6afb9a7ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.188 7f6afb9a7ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:20.188 7f6afb9a7ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.211 7f6afbb2fc20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:32:20.212 7f6afd1ac2c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.213 7f6afd1ac2c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:20.213 7f6afd1ac2c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:20.234 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:20.235 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.235 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:20.235 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.235 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:20.236 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.496 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:20.496 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.497 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:20.497 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.497 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:20.498 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.985 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:20.986 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.986 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:20.986 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:20.987 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:20.987 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:21.052 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:22.054 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:22.103 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:22.104 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:22.104 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:22.105 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:22.105 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:22.105 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:23.054 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:24.049 7f6afd1ac7c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:24.050 7f6afd1ac7c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:24.050 7f6afd1ac7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:24.050 7f6afd1ac7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:24.051 7f6afd1ac7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:24.051 7f6afd1ac7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:24.051 7f6afd1ac7c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:24.051 7f6afd1ac7c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:32:24.051 7f6afd1ac7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:24.051 7f6afd1ac7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:24.051 7f6afd1ac7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:24.051 7f6afd1aca40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:32:24.052 7f6afd1aca40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:32:24.055 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:24.942 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:24.943 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:24.943 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:24.943 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:24.943 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:24.944 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:25.056 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:26.057 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:27.058 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:28.058 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:29.059 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:30.061 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:30.138 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:30.139 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:30.139 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:30.140 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:30.140 7f6afbadfc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:30.140 7f6afbadfc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:31.062 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:32.063 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:33.064 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:34.065 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:35.061 7f6afbadfc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:32:35.061 7f6afbadfc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:32:35.062 7f6afbadfc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:35.067 7f6afbadf2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:35.217 7f6afbb2fc20 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:32:35.218 7f6b008ea8c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:32:35.218 7f6b008ea8c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:32:35.218 7f6afbadfcc0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:32:35.218 7f6afbadfcc0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:32:35.231 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:32:35.232 7f6b008ea8c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:32:43.741 7f5395b0a240 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:32:43.741 7f5395b0a240 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:32:43.741 7f5395b0a240 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:32:43.744 7f5395b0a240 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:32:43.745 7f53910522c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:32:43.745 7f53910522c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:32:43.745 7f53910527c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:32:43.745 7f53910527c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:32:43.746 7f53910527c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:32:43.746 7f53910527c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:32:43.746 7f53910527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.746 7f53910527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:32:43.746 7f53910527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.746 7f53910527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.746 7f53910527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353163745986 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:32:43.746 7f53910527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.747 7f53910527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.747 7f53910527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:43.747 7f53910527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.748 7f53910527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.748 7f53910527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:43.748 7f53910527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.748 7f53910527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.748 7f53910527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:32:43.748 7f53910527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.748 7f53910527c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.748 7f53910527c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:43.748 7f53910527c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.748 7f5391052e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:43.748 7f5391052e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:32:43.749 7f5391052c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:32:43.750 7f5391002540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.750 7f5391002540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:32:43.750 7f5391002540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.750 7f5391002540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.750 7f5391002540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:43.750 7f5391002540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.801 7f53910527c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:32:43.808 7f53910527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:32:43.809 7f5390f960e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.809 7f5390f960e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:43.809 7f5390f960e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.809 7f5390f960e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.809 7f5390f960e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:43.809 7f5390f960e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.810 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:43.811 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:43.811 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:43.812 7f53910027c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.812 7f53910027c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:43.812 7f53910027c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.812 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:43.813 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:43.813 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:43.814 7f53910527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:32:43.814 7f53910027c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.814 7f53910027c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:43.814 7f53910027c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.815 7f53910027c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.815 7f53910027c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:43.815 7f53910027c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.821 7f5390f96d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.822 7f5390f96d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:43.822 7f5390f96d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.851 7f53910527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:43.851 7f5390f96c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.852 7f5390f96c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:32:43.852 7f5390f96c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.855 7f53910527c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:43.855 7f5390f96c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.855 7f5390f96c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:32:43.855 7f5390f96c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.856 7f53910527c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:32:43.856 7f53910527c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:32:43.856 7f53910527c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:32:43.857 7f5390f96f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:32:43.857 7f5390ed4220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.857 7f5390ed4220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:43.857 7f5390ed4220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.857 7f5390ed4220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.857 7f5390ed4220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:43.857 7f5390ed4220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.860 7f53910527c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:32:43.863 7f5390f96f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:32:43.863 7f5390ed4ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.863 7f5390ed4ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:32:43.863 7f5390ed4ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.886 7f53910527c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:32:43.888 7f53926cf400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.888 7f53926cf400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:43.888 7f53926cf400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:43.910 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:43.911 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:43.911 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:43.912 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:43.912 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:43.912 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:44.089 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:44.090 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:44.090 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:44.090 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:44.090 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:44.091 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:44.385 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:44.386 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:44.386 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:44.386 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:44.387 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:44.387 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:44.750 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:44.984 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:44.985 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:44.985 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:44.986 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:44.986 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:44.986 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:45.751 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:46.007 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:46.008 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:46.008 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:46.008 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:46.008 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:46.009 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:46.752 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:47.746 7f53926cf400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:47.747 7f53926cf400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:47.747 7f53926cf400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:47.747 7f53926cf400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:47.747 7f53926cf400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:32:47.748 7f53926cf400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:47.748 7f53926cf400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:47.748 7f53926cf400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:32:47.748 7f53926cf400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:47.748 7f53926cf400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:47.748 7f53926cf400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:47.748 7f53926cf900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:32:47.749 7f53926cf900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:32:47.754 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:48.419 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:48.420 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.420 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:48.420 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.420 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:48.421 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:48.755 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:49.756 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:50.758 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:51.759 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:52.760 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:52.831 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:32:52.831 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:52.832 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:32:52.832 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:52.832 7f5391002c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:32:52.832 7f5391002c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:32:53.762 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:54.763 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:55.764 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:56.765 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:57.766 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:58.755 7f5391002c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:32:58.756 7f5391002c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:32:58.756 7f53917f3c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:58.767 7f5391052f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:32:58.900 7f53910527c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:32:58.901 7f5395b0a240 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:32:58.901 7f5395b0a240 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:32:58.901 7f5391002cc0 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:32:58.901 7f5391002cc0 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:32:58.915 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:32:58.915 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:32:58.915 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:32:58.915 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:32:58.915 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:32:58.915 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:32:58.915 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:32:58.916 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:32:58.916 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:32:58.916 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:32:58.916 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:32:58.916 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:32:58.916 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:32:58.916 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:32:58.916 7f5395b0a240 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:33:07.358 7f374b254cc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:33:07.358 7f374b254cc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:33:07.358 7f374b254cc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:33:07.361 7f374b254cc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:33:07.362 7f37467c12c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:33:07.362 7f37467c12c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:33:07.362 7f37467c17c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:33:07.363 7f37467c17c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:33:07.363 7f37467c17c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:33:07.363 7f37467c17c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:33:07.363 7f37467c17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.363 7f37467c17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:33:07.363 7f37467c17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.363 7f37467c17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.364 7f37467c17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353187363443 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:33:07.364 7f37467c17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.365 7f37467c1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:07.366 7f37467c1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.366 7f37467c1f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:07.366 7f37467c1f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:33:07.368 7f37467c1e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:33:07.368 7f37467717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.368 7f37467717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:07.368 7f37467717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.368 7f37467717c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.368 7f37467717c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:07.369 7f37467717c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.440 7f37467c1c20 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:33:07.450 7f37467c1c20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:33:07.451 7f37467030e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.451 7f37467030e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:07.451 7f37467030e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.451 7f37467030e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.451 7f37467030e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:07.451 7f37467030e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.454 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:07.455 7f37467719a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.456 7f37467719a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:07.456 7f37467719a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.457 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:07.457 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:07.457 7f37467c1c20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:33:07.457 7f37467719a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.458 7f37467719a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:07.458 7f37467719a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.459 7f37467030e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.463 7f37467030e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:07.463 7f37467030e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.471 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:07.471 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:07.472 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:07.473 7f3746703d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.473 7f3746703d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:07.473 7f3746703d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.493 7f37467c1c20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:07.494 7f3746703c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.494 7f3746703c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:07.495 7f3746703c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.498 7f37467c1c20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:07.498 7f3746703c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.498 7f3746703c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:07.498 7f3746703c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.499 7f37467c1c20 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:33:07.499 7f37467c1c20 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:33:07.499 7f37467c1c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:33:07.499 7f3746703f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:07.500 7f3746641220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.500 7f3746641220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:07.500 7f3746641220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.500 7f3746641220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.500 7f3746641220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:07.500 7f3746641220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.503 7f37467c1c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:33:07.505 7f3746703f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:07.505 7f37466419a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.505 7f37466419a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:07.505 7f37466419a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.529 7f37467c1c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:33:07.531 7f3747e52180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.531 7f3747e52180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:07.531 7f3747e52180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:07.595 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:07.595 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:07.595 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:07.596 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:07.596 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:07.597 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:07.897 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:07.898 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:07.898 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:07.898 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:07.898 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:07.899 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:08.307 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:08.308 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:08.308 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:08.308 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:08.308 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:08.309 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:08.368 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:09.348 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:09.349 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:09.349 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:09.350 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:09.350 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:09.350 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:09.369 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:10.370 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:11.363 7f3747e52680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:11.365 7f3747e52680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:11.365 7f3747e52680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:11.365 7f3747e52680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:11.365 7f3747e52680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:11.365 7f3747e52680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:11.365 7f3747e52680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:11.365 7f3747e52680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:33:11.365 7f3747e52680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:11.365 7f3747e52680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:11.366 7f3747e52680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:11.366 7f3747e52900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:33:11.367 7f3747e52900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:33:11.370 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:11.704 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:11.705 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:11.705 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:11.706 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:11.706 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:11.706 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:12.371 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:13.373 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:14.374 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:15.376 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:16.377 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:16.424 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:16.425 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:16.425 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:16.426 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:16.426 7f3746771c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:16.427 7f3746771c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:17.379 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:18.379 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:19.380 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:20.382 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:21.382 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:22.383 7f37467712c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:22.455 7f3746771c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:33:22.456 7f3746771c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:33:22.456 7f3746771c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:22.624 7f37467c1c20 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:33:22.625 7f374b254cc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:33:22.625 7f374b254cc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:33:22.625 7f3746771860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:33:22.625 7f3746771860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:33:22.640 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:33:22.640 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:33:22.640 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:33:22.640 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:33:22.640 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:33:22.641 7f374b254cc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:33:31.497 7ff055991bc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:33:31.497 7ff055991bc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:33:31.497 7ff055991bc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:33:31.500 7ff055991bc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:33:31.501 7ff05092f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:33:31.501 7ff05092f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:33:31.501 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:33:31.502 7ff05092f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:33:31.502 7ff05092f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:33:31.502 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:33:31.502 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.502 7ff05092f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:33:31.502 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.502 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.503 7ff05092f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353211502390 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:33:31.503 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.503 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.503 7ff05092f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:31.504 7ff05092f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.504 7ff05092fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:31.505 7ff05092fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:33:31.505 7ff05092fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:33:31.506 7ff0508df540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.506 7ff0508df540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:31.506 7ff0508df540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.506 7ff0508df540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.506 7ff0508df540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:31.506 7ff0508df540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.570 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:33:31.579 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:33:31.580 7ff050878040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.581 7ff050878040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:31.581 7ff050878040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.581 7ff050878040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.581 7ff050878040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:31.581 7ff050878040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.582 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:31.583 7ff0508df7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.584 7ff0508df7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:31.584 7ff0508df7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.584 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:31.585 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:31.585 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:31.585 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:31.586 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:33:31.586 7ff050878040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.586 7ff050878040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:31.587 7ff050878040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.594 7ff050878d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.595 7ff050878d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:31.595 7ff050878d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.595 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:31.597 7ff050878d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.597 7ff050878d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:31.597 7ff050878d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.625 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:31.625 7ff050878c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.626 7ff050878c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:31.626 7ff050878c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.633 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:31.633 7ff050878c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.634 7ff050878c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:31.634 7ff050878c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.637 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:33:31.637 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:33:31.637 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:33:31.637 7ff050878f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:31.638 7ff0507b4220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.638 7ff0507b4220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:31.638 7ff0507b4220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.638 7ff0507b4220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.638 7ff0507b4220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:31.638 7ff0507b4220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.641 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:33:31.644 7ff050878f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:31.644 7ff0507b49a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.644 7ff0507b49a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:31.644 7ff0507b49a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.674 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:31.674 7ff05092f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:33:31.675 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:31.675 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:31.677 7ff051fc22c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.677 7ff051fc22c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:31.677 7ff051fc22c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:31.677 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:31.677 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:31.678 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:31.827 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:31.828 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:31.828 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:31.828 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:31.828 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:31.829 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:32.061 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:32.062 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:32.062 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:32.062 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:32.063 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:32.063 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:32.506 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:32.700 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:32.700 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:32.701 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:32.701 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:32.701 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:32.702 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:33.507 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:33.935 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:33.936 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:33.937 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:33.937 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:33.938 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:33.938 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:34.508 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:35.503 7ff051fc27c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:35.504 7ff051fc27c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:35.505 7ff051fc27c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:35.505 7ff051fc27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:35.505 7ff051fc27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:35.505 7ff051fc27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:35.505 7ff051fc27c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:35.505 7ff051fc27c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:33:35.506 7ff051fc27c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:35.506 7ff051fc27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:35.506 7ff051fc27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:35.506 7ff051fc2a40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:33:35.508 7ff051fc2a40 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:33:35.509 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:36.510 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:37.368 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:37.369 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.369 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:37.369 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.370 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:37.370 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:37.511 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:38.512 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:39.514 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:40.515 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:41.517 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:42.147 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:42.148 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:42.148 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:42.148 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:42.149 7ff0508dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:42.149 7ff0508dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:42.518 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:43.519 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:44.520 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:45.522 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:46.523 7ff05092ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:46.570 7ff0508dfa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:33:46.570 7ff0508dfa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:33:46.571 7ff0508dfa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:46.748 7ff05092f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:33:46.748 7ff055991bc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:33:46.748 7ff055991bc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:33:46.748 7ff0508df680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:33:46.749 7ff0508df680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:33:46.763 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:33:46.763 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:33:46.763 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:33:46.763 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:33:46.763 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:33:46.763 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:33:46.764 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:33:46.764 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:33:46.764 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:33:46.764 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:33:46.764 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:33:46.764 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:33:46.764 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:33:46.764 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:33:46.764 7ff055991bc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:33:56.189 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:33:56.189 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:33:56.189 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:33:56.193 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:33:56.194 7f9dd5a582c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:33:56.194 7f9dd5a582c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:33:56.194 7f9dd5a587c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:33:56.194 7f9dd5a587c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:33:56.195 7f9dd5a587c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:33:56.195 7f9dd5a587c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:33:56.195 7f9dd5a587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.195 7f9dd5a587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:33:56.195 7f9dd5a587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.195 7f9dd5a587c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.195 7f9dd5a587c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353236195227 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:33:56.196 7f9dd5a587c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.197 7f9dd5a58c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:56.198 7f9dd5a58c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.198 7f9dd5a58f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:56.198 7f9dd5a58f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:33:56.199 7f9dd5a58e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:33:56.200 7f9dd5a087c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.200 7f9dd5a087c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:33:56.200 7f9dd5a087c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.200 7f9dd5a087c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.200 7f9dd5a087c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:56.200 7f9dd5a087c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.255 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:33:56.270 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:33:56.271 7f9dd7b690e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.271 7f9dd7b690e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:56.271 7f9dd7b690e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.271 7f9dd7b690e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.271 7f9dd7b690e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:56.271 7f9dd7b690e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.272 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:56.273 7f9dd5a089a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.274 7f9dd5a089a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:56.274 7f9dd5a089a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.274 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.274 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:56.275 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.275 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:56.276 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:33:56.276 7f9dd5a089a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.276 7f9dd5a089a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:56.276 7f9dd5a089a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.277 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.283 7f9dd7b69d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.283 7f9dd7b69d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:56.283 7f9dd7b69d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.285 7f9dd7b69d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.285 7f9dd7b69d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:56.285 7f9dd7b69d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.307 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:56.308 7f9dd7b69c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.308 7f9dd7b69c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:33:56.308 7f9dd7b69c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.311 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:56.312 7f9dd7b69c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.312 7f9dd7b69c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:33:56.312 7f9dd7b69c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.312 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:33:56.313 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:33:56.313 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:33:56.313 7f9dd7b69f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:33:56.313 7f9dd702b220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.314 7f9dd702b220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:56.314 7f9dd702b220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.314 7f9dd702b220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.314 7f9dd702b220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:56.314 7f9dd702b220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.317 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:33:56.320 7f9dd7b69f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:33:56.320 7f9dd702b9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.320 7f9dd702b9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:33:56.322 7f9dd702b9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.349 7f9dd5a58c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:33:56.350 7f9dd7009400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.350 7f9dd7009400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:33:56.350 7f9dd7009400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:56.408 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:56.409 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.409 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:56.409 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.411 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:56.411 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.591 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:56.591 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.591 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:56.592 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.592 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:56.593 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.904 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:56.904 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.904 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:56.905 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:56.905 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:56.906 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:57.203 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:57.422 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:57.423 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:57.423 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:57.424 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:57.424 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:57.424 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:58.205 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:33:58.850 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:33:58.851 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:58.851 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:33:58.851 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:58.851 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:33:58.852 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:33:59.206 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:00.195 7f9dd702b900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:00.196 7f9dd702b900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:00.196 7f9dd702b900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:00.196 7f9dd702b900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:00.197 7f9dd702b900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:00.197 7f9dd702b900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:00.197 7f9dd702b900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:00.197 7f9dd702b900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:34:00.197 7f9dd702b900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:00.198 7f9dd702b900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:00.198 7f9dd702b900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:00.198 7f9dd702b7c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:34:00.199 7f9dd702b7c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:34:00.207 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:01.208 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:02.209 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:02.346 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:02.347 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:02.347 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:02.347 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:02.347 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:02.348 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:03.211 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:04.212 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:05.213 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:06.215 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:07.216 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:08.217 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:09.219 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:10.220 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:10.824 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:10.825 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:10.825 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:10.826 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:10.826 7f9dd5a08c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:10.826 7f9dd5a08c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:11.221 7f9dd5a082c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:11.228 7f9dd5a08c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:34:11.228 7f9dd5a08c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:34:11.228 7f9dd5a08c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:11.397 7f9dd5a58c20 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:34:11.397 7f9ddac0ee00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:34:11.397 7f9ddac0ee00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:34:11.397 7f9dd5a08b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:34:11.397 7f9dd5a08b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:34:11.413 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:34:11.414 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:34:11.414 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:34:11.414 7f9ddac0ee00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:34:20.323 7f4428113a80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:34:20.323 7f4428113a80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:34:20.323 7f4428113a80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:34:20.326 7f4428113a80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:34:20.327 7f44230f52c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:34:20.327 7f44230f52c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:34:20.327 7f44230f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:34:20.327 7f44230f57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:20.328 7f44230f57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:34:20.328 7f44230f57c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:34:20.328 7f44230f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.328 7f44230f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:34:20.328 7f44230f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.328 7f44230f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.328 7f44230f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353260328037 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:34:20.328 7f44230f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.329 7f44230f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.329 7f44230f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:20.329 7f44230f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.329 7f44230f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.329 7f44230f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:20.329 7f44230f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.329 7f44230f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.329 7f44230f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:20.330 7f44230f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.330 7f44230f57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.330 7f44230f57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:20.330 7f44230f57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.330 7f44230f5e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:20.330 7f44230f5e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:34:20.331 7f44230f5c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:34:20.332 7f44230a4540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.332 7f44230a4540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:20.332 7f44230a4540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.332 7f44230a4540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.332 7f44230a4540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:20.332 7f44230a4540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.388 7f44230f57c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:34:20.397 7f44230f57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:34:20.398 7f442583b040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.398 7f442583b040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:20.398 7f442583b040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.398 7f442583b040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.398 7f442583b040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:20.399 7f442583b040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.400 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:20.400 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:20.400 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:20.401 7f44230a47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.401 7f44230a47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:20.402 7f44230a47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.402 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:20.402 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:20.403 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:20.404 7f442583b040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.405 7f442583b040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:20.405 7f442583b040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.405 7f44230f57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:34:20.406 7f442583b040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.406 7f442583b040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:20.406 7f442583b040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.412 7f442583bd60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.413 7f442583bd60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:20.413 7f442583bd60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.443 7f44230f57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:20.444 7f442583bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.444 7f442583bc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:20.444 7f442583bc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.447 7f44230f57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:20.447 7f442583bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.447 7f442583bc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:20.447 7f442583bc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.447 7f44230f57c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:34:20.448 7f44230f57c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:34:20.448 7f44230f57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:34:20.448 7f442583bf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:20.449 7f4422f80220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.449 7f4422f80220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:20.449 7f4422f80220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.449 7f4422f80220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.449 7f4422f80220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:20.449 7f4422f80220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.453 7f44230f57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:34:20.455 7f442583bf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:20.455 7f4422f809a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.455 7f4422f809a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:20.455 7f4422f809a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.481 7f44230f57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:34:20.482 7f44246cb180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.483 7f44246cb180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:20.483 7f44246cb180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:20.538 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:20.539 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:20.539 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:20.540 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:20.540 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:20.540 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:20.782 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:20.783 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:20.783 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:20.784 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:20.784 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:20.785 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:21.332 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:21.340 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:21.341 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:21.341 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:21.342 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:21.342 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:21.343 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:22.333 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:22.463 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:22.464 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:22.464 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:22.465 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:22.465 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:22.465 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:23.334 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:23.932 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:23.932 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:23.932 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:23.933 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:23.933 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:23.934 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:24.328 7f44246cb680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:24.330 7f44246cb680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:24.330 7f44246cb680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:24.330 7f44246cb680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:24.330 7f44246cb680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:24.330 7f44246cb680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:24.330 7f44246cb680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:24.330 7f44246cb680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:34:24.331 7f44246cb680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:24.331 7f44246cb680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:24.331 7f44246cb680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:24.331 7f44246cb900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:34:24.332 7f44246cb900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:34:24.337 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:25.338 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:26.339 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:27.141 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:27.142 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:27.142 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:27.143 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:27.143 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:27.143 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:27.340 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:28.341 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:29.343 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:30.344 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:31.346 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:32.347 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:33.348 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:34.350 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:35.112 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:35.113 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:35.113 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:35.113 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:35.113 7f44230a4a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:35.114 7f44230a4a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:35.351 7f44230f5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:35.415 7f44230a4a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:34:35.415 7f44230a4a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:34:35.416 7f44230a4a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:35.499 7f44230f57c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:34:35.500 7f4428113a80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:34:35.500 7f4428113a80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:34:35.500 7f44230a4680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:34:35.500 7f44230a4680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:34:35.515 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:34:35.515 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:34:35.515 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:34:35.515 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:34:35.515 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:34:35.515 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:34:35.515 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:34:35.515 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:34:35.516 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:34:35.516 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:34:35.516 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:34:35.516 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:34:35.516 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:34:35.516 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:34:35.516 7f4428113a80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:34:44.605 7f47893af400 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:34:44.605 7f47893af400 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:34:44.605 7f47893af400 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:34:44.609 7f47893af400 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:34:44.609 7f47848d22c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:34:44.609 7f47848d22c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:34:44.610 7f47848d27c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:34:44.610 7f47848d27c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:34:44.610 7f47848d27c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:34:44.611 7f47848d27c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:34:44.611 7f47848d27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.611 7f47848d27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:34:44.611 7f47848d27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.611 7f47848d27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.611 7f47848d27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353284610836 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:34:44.611 7f47848d27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.612 7f47848d27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.612 7f47848d27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:44.612 7f47848d27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.612 7f47848d27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.612 7f47848d27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:44.612 7f47848d27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.612 7f47848d27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.613 7f47848d27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:34:44.613 7f47848d27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.613 7f47848d27c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.613 7f47848d27c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:44.613 7f47848d27c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.613 7f47848d2e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:44.613 7f47848d2e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:34:44.615 7f47848d2c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:34:44.615 7f4784882540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.615 7f4784882540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:34:44.616 7f4784882540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.616 7f4784882540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.616 7f4784882540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:44.616 7f4784882540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.678 7f47848d27c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:34:44.686 7f47848d27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:34:44.687 7f47848620e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.688 7f47848620e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:44.688 7f47848620e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.688 7f47848620e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.688 7f47848620e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:44.688 7f47848620e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.689 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:44.690 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:44.690 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:44.691 7f47848827c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.692 7f47848827c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:44.692 7f47848827c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.693 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:44.693 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:44.694 7f47848d27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:34:44.695 7f47848827c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.695 7f47848827c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:44.695 7f47848827c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.695 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:44.696 7f47848620e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.696 7f47848620e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:44.696 7f47848620e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.705 7f4784862d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.705 7f4784862d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:44.705 7f4784862d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.737 7f47848d27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:44.738 7f4784862c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.738 7f4784862c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:34:44.738 7f4784862c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.742 7f47848d27c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:44.742 7f4784862c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.742 7f4784862c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:34:44.742 7f4784862c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.743 7f47848d27c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:34:44.743 7f47848d27c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:34:44.743 7f47848d27c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:34:44.743 7f4784862f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:34:44.744 7f4785ea9220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.744 7f4785ea9220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:44.744 7f4785ea9220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.745 7f4785ea9220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.745 7f4785ea9220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:44.745 7f4785ea9220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.749 7f47848d27c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:34:44.751 7f4784862f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:34:44.751 7f4785ea99a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.751 7f4785ea99a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:34:44.751 7f4785ea99a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.801 7f47848d27c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:34:44.805 7f4785ea9900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.805 7f4785ea9900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:44.805 7f4785ea9900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:44.806 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:44.807 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:44.807 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:44.809 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:44.809 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:44.810 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:45.062 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:45.063 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:45.063 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:45.064 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:45.064 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:45.064 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:45.396 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:45.396 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:45.397 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:45.397 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:45.397 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:45.398 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:45.615 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:45.821 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:45.822 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:45.822 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:45.823 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:45.823 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:45.823 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:46.616 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:46.799 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:46.800 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:46.800 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:46.801 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:46.801 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:46.801 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:47.617 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:48.611 7f4785ea9900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:48.613 7f4785ea9900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:48.613 7f4785ea9900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:48.613 7f4785ea9900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:48.613 7f4785ea9900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:34:48.613 7f4785ea9900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:48.613 7f4785ea9900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:48.613 7f4785ea9900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:34:48.614 7f4785ea9900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:48.614 7f4785ea9900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:48.614 7f4785ea9900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:48.614 7f47841e8180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:34:48.615 7f47841e8180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:34:48.622 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:49.154 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:49.155 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:49.155 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:49.155 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:49.155 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:49.156 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:49.624 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:50.624 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:51.625 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:52.626 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:53.628 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:53.656 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:34:53.657 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:53.657 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:34:53.657 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:53.657 7f4784882c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:34:53.658 7f4784882c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:34:54.628 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:55.629 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:56.630 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:57.631 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:58.632 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:59.633 7f47848d2f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:34:59.686 7f4784882c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:34:59.687 7f4784882c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:34:59.687 7f4785073c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:59.875 7f47848d27c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:34:59.875 7f47893af400 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:34:59.875 7f47893af400 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:34:59.875 7f4784882680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:34:59.875 7f4784882680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:34:59.889 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:34:59.890 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:34:59.890 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:34:59.890 7f47893af400 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:35:08.386 7fdca0e72800 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:35:08.386 7fdca0e72800 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:35:08.386 7fdca0e72800 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:35:08.389 7fdca0e72800 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:35:08.390 7fdc9c0b52c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:35:08.390 7fdc9c0b52c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:35:08.390 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:35:08.391 7fdc9c0b57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:35:08.391 7fdc9c0b57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:35:08.391 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:35:08.392 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.392 7fdc9c0b57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:35:08.392 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.392 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.392 7fdc9c0b57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353308391611 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:35:08.392 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.393 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.394 7fdc9c0b57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:08.394 7fdc9c0b57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.394 7fdc9c0b5e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:08.394 7fdc9c0b5e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:35:08.395 7fdc9c0b5c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:35:08.396 7fdc9c065540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.396 7fdc9c065540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:08.396 7fdc9c065540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.396 7fdc9c065540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.396 7fdc9c065540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:08.396 7fdc9c065540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.454 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:35:08.462 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:35:08.462 7fdc9e1c5040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.463 7fdc9e1c5040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:08.463 7fdc9e1c5040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.463 7fdc9e1c5040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.463 7fdc9e1c5040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:08.463 7fdc9e1c5040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.464 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:08.465 7fdc9c0657c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.465 7fdc9c0657c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:08.466 7fdc9c0657c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.466 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:08.466 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:08.467 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:08.467 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:08.468 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:35:08.468 7fdc9c0657c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.468 7fdc9c0657c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:08.468 7fdc9c0657c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.469 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:08.469 7fdc9e1c5040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.470 7fdc9e1c5040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:08.470 7fdc9e1c5040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.478 7fdc9e1c5c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.479 7fdc9e1c5c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:08.479 7fdc9e1c5c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.510 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:08.510 7fdc9e1c5ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.511 7fdc9e1c5ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:08.511 7fdc9e1c5ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.514 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:08.514 7fdc9e1c5ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.514 7fdc9e1c5ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:08.514 7fdc9e1c5ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.515 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:35:08.515 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:35:08.515 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:35:08.515 7fdc9e1c5e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:08.516 7fdc9c03e0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.516 7fdc9c03e0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:08.516 7fdc9c03e0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.516 7fdc9c03e0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.516 7fdc9c03e0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:08.516 7fdc9c03e0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.520 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:35:08.521 7fdc9e1c5e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:08.522 7fdc9c03e860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.522 7fdc9c03e860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:08.522 7fdc9c03e860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.547 7fdc9c0b57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:35:08.549 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.549 7fdc9d67e400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:08.549 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:08.592 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:08.592 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:08.592 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:08.593 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:08.593 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:08.594 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:08.824 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:08.825 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:08.825 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:08.825 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:08.826 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:08.826 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:09.269 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:09.269 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:09.270 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:09.270 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:09.270 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:09.271 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:09.396 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:10.357 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:10.357 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:10.358 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:10.358 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:10.358 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:10.359 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:10.397 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:11.398 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:12.256 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:12.257 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:12.257 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:12.258 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:12.258 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:12.258 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:12.391 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:12.395 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:12.395 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:12.395 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:12.396 7fdc9d67e400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:12.396 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:12.396 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:12.396 7fdc9d67e400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:35:12.396 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:12.397 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:12.397 7fdc9d67e400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:12.397 7fdc9d67e900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:35:12.398 7fdc9d67e900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:35:12.399 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:13.400 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:14.401 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:15.402 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:15.409 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:15.410 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:15.410 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:15.411 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:15.411 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:15.411 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:16.404 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:17.405 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:18.406 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:19.407 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:20.408 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:20.831 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:20.831 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:20.832 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:20.832 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:20.833 7fdc9c065a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:20.833 7fdc9c065a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:21.410 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:22.411 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:23.412 7fdc9c0b5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:23.446 7fdc9c065a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:35:23.446 7fdc9c065a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:35:23.447 7fdc9c065a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:23.587 7fdc9c0b57c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:35:23.587 7fdca0e72800 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:35:23.587 7fdca0e72800 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:35:23.587 7fdc9c065680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:35:23.587 7fdc9c065680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:35:23.609 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:35:23.610 7fdca0e72800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:35:32.528 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:35:32.528 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:35:32.528 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:35:32.532 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:35:32.532 7f4afc58c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:35:32.533 7f4afc58c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:35:32.533 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:35:32.533 7f4afc58c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:35:32.534 7f4afc58c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:35:32.534 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:35:32.534 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.535 7f4afc58c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:35:32.535 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.535 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.535 7f4afc58c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353332534317 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:35:32.535 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.536 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.536 7f4afc58c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:32.536 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.536 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.536 7f4afc58c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:32.536 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.536 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.536 7f4afc58c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:32.536 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.537 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.537 7f4afc58c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:32.537 7f4afc58c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.537 7f4afc58cf40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:32.537 7f4afc58cf40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:35:32.538 7f4afc58ce00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:35:32.539 7f4afc53c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.539 7f4afc53c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:32.539 7f4afc53c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.539 7f4afc53c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.539 7f4afc53c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:32.539 7f4afc53c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.605 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:35:32.614 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:35:32.615 7f4afc4950e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.615 7f4afc4950e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:32.615 7f4afc4950e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.615 7f4afc4950e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.615 7f4afc4950e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:32.615 7f4afc4950e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.616 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:32.618 7f4afc53c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.618 7f4afc53c9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:32.618 7f4afc53c9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.618 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:32.618 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:32.620 7f4afc53c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.620 7f4afc53c9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:32.621 7f4afc53c9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.621 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:32.621 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:32.621 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:35:32.622 7f4afc53c9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.622 7f4afc53c9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:32.622 7f4afc53c9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.624 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:32.632 7f4afc495d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.632 7f4afc495d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:32.633 7f4afc495d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.664 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:32.665 7f4afc495c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.665 7f4afc495c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:32.665 7f4afc495c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.668 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:32.668 7f4afc495c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.668 7f4afc495c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:32.669 7f4afc495c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.669 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:35:32.669 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:35:32.669 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:35:32.669 7f4afc495f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:32.670 7f4afc40d220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.670 7f4afc40d220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:32.670 7f4afc40d220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.670 7f4afc40d220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.670 7f4afc40d220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:32.671 7f4afc40d220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.675 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:35:32.677 7f4afc495f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:32.677 7f4afc40d9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.677 7f4afc40d9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:32.677 7f4afc40d9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.705 7f4afc58c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:35:32.706 7f4afc40d900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.707 7f4afc40d900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:32.707 7f4afc40d900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:32.710 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:32.711 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:32.711 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:32.711 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:32.711 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:32.712 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:32.832 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:32.833 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:32.833 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:32.833 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:32.833 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:32.834 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:33.136 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:33.137 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:33.137 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:33.138 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:33.138 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:33.138 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:33.538 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:33.789 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:33.790 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:33.790 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:33.791 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:33.791 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:33.791 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:34.539 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:35.275 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:35.276 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:35.276 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:35.276 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:35.276 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:35.277 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:35.541 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:36.534 7f4afc40d900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:36.535 7f4afc40d900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:36.535 7f4afc40d900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:36.535 7f4afc40d900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:36.536 7f4afc40d900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:36.536 7f4afc40d900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:36.536 7f4afc40d900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:36.536 7f4afc40d900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:35:36.536 7f4afc40d900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:36.537 7f4afc40d900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:36.537 7f4afc40d900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:36.537 7f4afec67180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:35:36.538 7f4afec67180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:35:36.542 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:37.542 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:37.567 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:37.568 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:37.568 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:37.568 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:37.568 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:37.569 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:38.544 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:39.545 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:40.546 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:41.547 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:41.834 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:41.835 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:41.835 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:41.835 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:41.835 7f4afc53cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:41.836 7f4afc53cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:42.549 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:43.550 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:44.551 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:45.553 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:46.554 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:47.555 7f4afc53c2c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:47.562 7f4afc53cc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:35:47.562 7f4afc53cc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:35:47.562 7f4afc53cc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:47.744 7f4afc58c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:35:47.744 7f4b0100cdc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:35:47.744 7f4b0100cdc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:35:47.744 7f4afc53cb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:35:47.744 7f4afc53cb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:35:47.767 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:35:47.768 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:35:47.768 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:35:47.768 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:35:47.768 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:35:47.768 7f4b0100cdc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:35:57.242 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:35:57.243 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:35:57.243 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:35:57.246 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:35:57.247 7ff9fac932c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:35:57.247 7ff9fac932c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:35:57.248 7ff9fac937c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:35:57.248 7ff9fac937c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:35:57.248 7ff9fac937c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:35:57.249 7ff9fac937c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:35:57.249 7ff9fac937c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.249 7ff9fac937c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:35:57.249 7ff9fac937c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.249 7ff9fac937c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.249 7ff9fac937c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353357248884 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:35:57.249 7ff9fac937c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.251 7ff9fac93c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.251 7ff9fac93c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:57.251 7ff9fac93c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.252 7ff9fac93c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.252 7ff9fac93c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:57.252 7ff9fac93c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.252 7ff9fac93c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.252 7ff9fac93c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:35:57.252 7ff9fac93c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.252 7ff9fac93c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.252 7ff9fac93c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:57.252 7ff9fac93c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.253 7ff9fac93f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:57.253 7ff9fac93f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:35:57.254 7ff9fac93e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:35:57.255 7ff9fac437c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.255 7ff9fac437c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:35:57.255 7ff9fac437c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.255 7ff9fac437c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.256 7ff9fac437c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:57.256 7ff9fac437c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.325 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:35:57.335 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:35:57.336 7ff9faba30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.336 7ff9faba30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:57.336 7ff9faba30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.336 7ff9faba30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.336 7ff9faba30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:57.336 7ff9faba30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.338 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:57.339 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:57.339 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:57.340 7ff9fac439a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.341 7ff9fac439a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:57.341 7ff9fac439a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.341 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:57.342 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:57.342 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:35:57.342 7ff9fac439a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.343 7ff9fac439a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:57.343 7ff9fac439a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.344 7ff9faba30e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.344 7ff9faba30e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:57.345 7ff9faba30e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.352 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:57.354 7ff9faba3d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.354 7ff9faba3d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:57.354 7ff9faba3d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.392 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:57.394 7ff9faba3c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.394 7ff9faba3c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:35:57.394 7ff9faba3c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.398 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:57.398 7ff9faba3c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.399 7ff9faba3c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:35:57.399 7ff9faba3c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.399 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:35:57.399 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:35:57.400 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:35:57.401 7ff9faba3f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:35:57.402 7ff9fab0d220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.402 7ff9fab0d220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:57.402 7ff9fab0d220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.403 7ff9fab0d220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.403 7ff9fab0d220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:57.403 7ff9fab0d220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.408 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:35:57.412 7ff9faba3f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:35:57.413 7ff9fab0d9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.413 7ff9fab0d9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:35:57.413 7ff9fab0d9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.451 7ff9fac93c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:35:57.453 7ff9fc329400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.453 7ff9fc329400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:35:57.453 7ff9fc329400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:57.475 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:57.476 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:57.476 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:57.476 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:57.477 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:57.477 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:57.672 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:57.673 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:57.673 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:57.674 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:57.674 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:57.675 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:58.150 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:58.151 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:58.151 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:58.152 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:58.152 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:58.152 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:58.254 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:35:58.992 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:35:58.993 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:58.993 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:35:58.993 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:58.993 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:35:58.994 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:35:59.255 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:00.256 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:00.318 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:00.319 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:00.319 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:00.320 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:00.320 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:00.320 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:01.249 7ff9fab0d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:01.250 7ff9fab0d7c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:01.251 7ff9fab0d7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:01.251 7ff9fab0d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:01.251 7ff9fab0d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:01.251 7ff9fab0d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:01.251 7ff9fab0d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:01.251 7ff9fab0d7c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:36:01.251 7ff9fab0d7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:01.252 7ff9fab0d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:01.252 7ff9fab0d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:01.252 7ff9fd370180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:36:01.253 7ff9fd370180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:36:01.257 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:02.259 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:03.260 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:03.896 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:03.896 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.897 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:03.897 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:03.897 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:03.897 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:04.261 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:05.263 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:06.264 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:07.265 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:08.266 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:09.267 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:10.269 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:11.271 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:12.010 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:12.011 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:12.011 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:12.012 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:12.012 7ff9fac43c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:12.012 7ff9fac43c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:12.272 7ff9fac432c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:12.314 7ff9fac43c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:36:12.314 7ff9fac43c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:36:12.315 7ff9fac43c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:12.475 7ff9fac93c20 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:36:12.475 7ff9ffcd9740 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:36:12.475 7ff9ffcd9740 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:36:12.475 7ff9fac43b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:36:12.475 7ff9fac43b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:36:12.488 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:36:12.488 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:36:12.488 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:36:12.488 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:36:12.489 7ff9ffcd9740 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:36:21.658 7fb6ebb57800 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:36:21.658 7fb6ebb57800 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:36:21.658 7fb6ebb57800 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:36:21.662 7fb6ebb57800 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:36:21.662 7fb6e6d9a2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:36:21.662 7fb6e6d9a2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:36:21.663 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:36:21.663 7fb6e6d9a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:36:21.663 7fb6e6d9a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:36:21.664 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:36:21.664 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.664 7fb6e6d9a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:36:21.664 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.664 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.664 7fb6e6d9a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353381663837 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:36:21.664 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.666 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.666 7fb6e6d9a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:21.666 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.666 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.666 7fb6e6d9a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:21.667 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.667 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.667 7fb6e6d9a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:21.667 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.667 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.667 7fb6e6d9a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:21.667 7fb6e6d9a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.668 7fb6e6d9ae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:21.668 7fb6e6d9ae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:36:21.670 7fb6e6d9ac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:36:21.671 7fb6e6d4a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.671 7fb6e6d4a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:21.671 7fb6e6d4a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.671 7fb6e6d4a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.671 7fb6e6d4a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:21.671 7fb6e6d4a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.742 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:36:21.752 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:36:21.753 7fb6e6d2b040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.753 7fb6e6d2b040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:21.753 7fb6e6d2b040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.753 7fb6e6d2b040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.754 7fb6e6d2b040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:21.754 7fb6e6d2b040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.755 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:21.756 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:21.756 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:21.757 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:21.757 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:21.758 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:21.759 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:36:21.759 7fb6e6d4a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.759 7fb6e6d4a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:21.759 7fb6e6d4a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.761 7fb6e6d4a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.761 7fb6e6d4a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:21.761 7fb6e6d4a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.771 7fb6e6d2bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.771 7fb6e6d2bc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:21.771 7fb6e6d2bc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.773 7fb6e6d2bc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.773 7fb6e6d2bc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:21.773 7fb6e6d2bc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.814 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:21.815 7fb6e6d2bae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.815 7fb6e6d2bae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:21.816 7fb6e6d2bae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.820 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:21.820 7fb6e6d2bae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.820 7fb6e6d2bae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:21.820 7fb6e6d2bae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.821 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:36:21.821 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:36:21.821 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:36:21.822 7fb6e6d2be00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:21.822 7fb6e666a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.823 7fb6e666a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:21.823 7fb6e666a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.823 7fb6e666a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.823 7fb6e666a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:21.823 7fb6e666a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.827 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:36:21.829 7fb6e6d2be00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:21.829 7fb6e666a9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.829 7fb6e666a9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:21.829 7fb6e666a9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.834 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:21.835 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:21.835 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:21.837 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:21.837 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:21.838 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:21.861 7fb6e6d9a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:36:21.863 7fb6e8ea4400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:21.864 7fb6e8ea4400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:21.864 7fb6e8ea4400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:22.042 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:22.043 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:22.043 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:22.043 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:22.044 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:22.044 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:22.441 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:22.442 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:22.443 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:22.443 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:22.443 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:22.444 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:22.669 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:23.016 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:23.017 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:23.017 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:23.017 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:23.017 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:23.018 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:23.671 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:24.567 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:24.567 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:24.568 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:24.568 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:24.568 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:24.569 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:24.672 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:25.664 7fb6e666a900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:25.665 7fb6e666a900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:25.665 7fb6e666a900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:25.665 7fb6e666a900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:25.666 7fb6e666a900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:25.666 7fb6e666a900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:25.666 7fb6e666a900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:25.666 7fb6e666a900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:36:25.666 7fb6e666a900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:25.667 7fb6e666a900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:25.667 7fb6e666a900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:25.667 7fb6e666a7c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:36:25.668 7fb6e666a7c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:36:25.673 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:26.674 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:27.675 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:28.464 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:28.465 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:28.465 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:28.466 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:28.466 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:28.466 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:28.676 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:29.678 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:30.679 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:31.681 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:32.682 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:33.658 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:33.659 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:33.659 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:33.660 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:33.660 7fb6e6d4aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:33.660 7fb6e6d4aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:33.684 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:34.685 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:35.688 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:36.674 7fb6e6d4aa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:36:36.674 7fb6e6d4aa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:36:36.675 7fb6e6d4aa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:36.689 7fb6e6d9af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:36.960 7fb6e6d9a7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:36:36.960 7fb6ebb57800 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:36:36.961 7fb6ebb57800 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:36:36.961 7fb6e6d4a860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:36:36.961 7fb6e6d4a860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:36:36.974 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:36:36.975 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:36:36.975 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:36:36.975 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:36:36.975 7fb6ebb57800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:36:45.835 7f3542735700 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:36:45.835 7f3542735700 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:36:45.835 7f3542735700 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:36:45.840 7f3542735700 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:36:45.841 7f353d9972c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:36:45.841 7f353d9972c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:36:45.841 7f353d9977c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:36:45.842 7f353d9977c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:36:45.842 7f353d9977c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:36:45.842 7f353d9977c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:36:45.843 7f353d9977c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.843 7f353d9977c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:36:45.843 7f353d9977c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.843 7f353d9977c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.843 7f353d9977c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353405842465 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:36:45.843 7f353d9977c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.844 7f353d9977c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.844 7f353d9977c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:45.844 7f353d9977c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.844 7f353d9977c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.844 7f353d9977c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:45.844 7f353d9977c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.845 7f353d9977c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.845 7f353d9977c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:36:45.845 7f353d9977c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.845 7f353d9977c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.845 7f353d9977c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:45.845 7f353d9977c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.845 7f353d997e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:45.846 7f353d997e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:36:45.847 7f353d997c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:36:45.848 7f353d947540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.848 7f353d947540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:36:45.848 7f353d947540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.848 7f353d947540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.848 7f353d947540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:45.848 7f353d947540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.907 7f353d9977c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:36:45.914 7f353d9977c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:36:45.915 7f353d8a6040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.915 7f353d8a6040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:45.915 7f353d8a6040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.915 7f353d8a6040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.915 7f353d8a6040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:45.915 7f353d8a6040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.916 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:45.917 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:45.917 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:45.918 7f353d9477c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.918 7f353d9477c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:45.918 7f353d9477c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.919 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:45.919 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:45.920 7f353d9477c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.920 7f353d9477c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:45.920 7f353d9477c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.921 7f353d9977c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:36:45.921 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:45.921 7f353d9477c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.921 7f353d9477c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:45.921 7f353d9477c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.928 7f353d8a6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.928 7f353d8a6c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:45.928 7f353d8a6c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.962 7f353d9977c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:45.963 7f353d8a6ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.963 7f353d8a6ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:36:45.963 7f353d8a6ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.967 7f353d9977c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:45.967 7f353d8a6ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.967 7f353d8a6ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:36:45.967 7f353d8a6ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.968 7f353d9977c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:36:45.968 7f353d9977c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:36:45.968 7f353d9977c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:36:45.968 7f353d8a6e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:36:45.969 7f353d8110e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.969 7f353d8110e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:45.969 7f353d8110e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.969 7f353d8110e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.970 7f353d8110e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:45.970 7f353d8110e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.973 7f353d9977c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:36:45.975 7f353d8a6e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:36:45.976 7f353d8119a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.976 7f353d8119a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:36:45.976 7f353d8119a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:45.986 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:46.000 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.001 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:46.001 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.002 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:46.003 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.006 7f353d9977c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:36:46.008 7f353d811900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:46.009 7f353d811900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:46.009 7f353d811900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:46.103 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:46.104 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.104 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:46.105 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.105 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:46.105 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.271 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:46.272 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.272 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:46.273 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.273 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:46.273 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.501 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:46.501 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.502 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:46.502 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.502 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:46.503 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.844 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:46.845 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.845 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:46.845 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.845 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:46.846 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:46.847 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:47.791 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:47.792 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:47.792 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:47.793 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:47.793 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:47.793 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:47.848 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:48.849 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:49.689 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:49.689 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:49.689 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:49.690 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:49.690 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:49.690 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:49.842 7f353d811900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:49.843 7f353d811900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:49.844 7f353d811900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:49.844 7f353d811900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:49.844 7f353d811900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:36:49.844 7f353d811900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:49.844 7f353d811900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:49.844 7f353d811900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:36:49.845 7f353d811900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:49.845 7f353d811900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:49.845 7f353d811900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:49.845 7f353fa9b180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:36:49.846 7f353fa9b180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:36:49.850 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:50.852 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:51.853 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:52.854 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:53.711 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:36:53.712 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:53.712 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:36:53.713 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:53.713 7f353d947a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:36:53.713 7f353d947a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:36:53.855 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:54.856 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:55.857 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:56.858 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:57.860 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:58.861 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:36:59.862 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:00.863 7f353d997f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:00.948 7f353d947a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:37:00.948 7f353d947a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:37:00.948 7f353d947a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:01.059 7f353d9977c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:37:01.059 7f3542735700 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:37:01.059 7f3542735700 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:37:01.060 7f353d947860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:37:01.060 7f353d947860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:37:01.130 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:37:01.130 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:37:01.130 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:37:01.130 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:37:01.131 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:37:01.132 7f3542735700 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:37:09.651 7f1984b35140 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:37:09.651 7f1984b35140 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:37:09.651 7f1984b35140 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:37:09.655 7f1984b35140 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:37:09.656 7f198009f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:37:09.656 7f198009f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:37:09.656 7f198009f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:37:09.657 7f198009f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:37:09.657 7f198009f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:37:09.657 7f198009f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:37:09.657 7f198009f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.658 7f198009f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:37:09.658 7f198009f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.658 7f198009f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.658 7f198009f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353429657643 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:37:09.658 7f198009f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.659 7f198009f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.660 7f198009f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:09.660 7f198009f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.660 7f198009fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:09.660 7f198009fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:37:09.661 7f198009fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:37:09.661 7f198004f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.662 7f198004f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:09.662 7f198004f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.662 7f198004f540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.662 7f198004f540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:09.662 7f198004f540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.719 7f198009f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:37:09.727 7f198009f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:37:09.728 7f197ffe8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.728 7f197ffe8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:09.728 7f197ffe8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.728 7f197ffe8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.728 7f197ffe8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:09.728 7f197ffe8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.729 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:09.730 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:09.730 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:09.731 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:09.731 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:09.732 7f198004f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.733 7f198004f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:09.733 7f198004f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.733 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:09.734 7f198009f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:37:09.734 7f198004f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.734 7f198004f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:09.734 7f198004f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.735 7f197ffe8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.735 7f197ffe8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:09.735 7f197ffe8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.742 7f197ffe8c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.743 7f197ffe8c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:09.743 7f197ffe8c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.772 7f198009f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:09.773 7f197ffe8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.773 7f197ffe8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:09.774 7f197ffe8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.776 7f198009f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:09.777 7f197ffe8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.777 7f197ffe8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:09.777 7f197ffe8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.777 7f198009f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:37:09.778 7f198009f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:37:09.778 7f198009f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:37:09.778 7f197ffe8e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:09.779 7f197ff190e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.779 7f197ff190e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:09.779 7f197ff190e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.779 7f197ff190e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.779 7f197ff190e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:09.779 7f197ff190e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.783 7f198009f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:37:09.786 7f197ffe8e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:09.786 7f197ff19860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.786 7f197ff19860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:09.786 7f197ff19860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.808 7f198009f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:37:09.810 7f1981731400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.811 7f1981731400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:09.811 7f1981731400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:09.850 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:09.850 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:09.850 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:09.851 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:09.851 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:09.851 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:10.081 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:10.081 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:10.082 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:10.082 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:10.082 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:10.083 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:10.617 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:10.618 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:10.618 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:10.619 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:10.619 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:10.619 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:10.661 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:11.662 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:11.987 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:11.988 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:11.988 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:11.989 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:11.989 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:11.989 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:12.663 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:13.657 7f1981731400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:13.659 7f1981731400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:13.659 7f1981731400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:13.659 7f1981731400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:13.659 7f1981731400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:13.660 7f1981731400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:13.660 7f1981731400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:13.660 7f1981731400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:37:13.660 7f1981731400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:13.660 7f1981731400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:13.660 7f1981731400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:13.661 7f1981731900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:37:13.662 7f1981731900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:37:13.664 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:14.555 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:14.556 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:14.556 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:14.557 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:14.557 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:14.557 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:14.665 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:15.667 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:16.668 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:17.669 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:18.475 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:18.476 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:18.476 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:18.476 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:18.477 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:18.477 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:18.674 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:19.676 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:20.677 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:21.678 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:22.679 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:23.681 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:24.345 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:24.346 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:24.346 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:24.347 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:24.347 7f198004fa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:24.347 7f198004fa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:24.682 7f198009ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:24.751 7f198004fa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:37:24.751 7f198004fa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:37:24.752 7f198004fa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:24.897 7f198009f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:37:24.897 7f1984b35140 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:37:24.898 7f1984b35140 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:37:24.898 7f198004f680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:37:24.898 7f198004f680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:37:24.922 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:37:24.922 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:37:24.922 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:37:24.922 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:37:24.923 7f1984b35140 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:37:33.758 7fc6dfe40600 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:37:33.759 7fc6dfe40600 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:37:33.759 7fc6dfe40600 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:37:33.762 7fc6dfe40600 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:37:33.763 7fc6db09d2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:37:33.763 7fc6db09d2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:37:33.764 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:37:33.764 7fc6db09d7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:37:33.764 7fc6db09d7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:37:33.765 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:37:33.765 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.765 7fc6db09d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:37:33.766 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.766 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.766 7fc6db09d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353453764689 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:37:33.766 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.767 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.767 7fc6db09d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:33.767 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.767 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.767 7fc6db09d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:33.768 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.768 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.768 7fc6db09d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:33.768 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.768 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.768 7fc6db09d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:33.768 7fc6db09d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.769 7fc6db09de00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:33.769 7fc6db09de00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:37:33.770 7fc6db09dc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:37:33.771 7fc6db04c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.772 7fc6db04c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:33.772 7fc6db04c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.772 7fc6db04c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.772 7fc6db04c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:33.772 7fc6db04c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.856 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:37:33.867 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:37:33.868 7fc6dafe60e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.869 7fc6dafe60e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:33.869 7fc6dafe60e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.869 7fc6dafe60e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.869 7fc6dafe60e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:33.869 7fc6dafe60e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.870 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:33.872 7fc6db04c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.872 7fc6db04c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:33.872 7fc6db04c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.873 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:33.873 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:33.874 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:33.874 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:33.875 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:37:33.876 7fc6db04c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.876 7fc6db04c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:33.876 7fc6db04c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.877 7fc6dafe60e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.877 7fc6dafe60e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:33.878 7fc6dafe60e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.878 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:33.888 7fc6dafe6d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.888 7fc6dafe6d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:33.888 7fc6dafe6d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.928 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:33.929 7fc6dafe6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.930 7fc6dafe6c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:33.930 7fc6dafe6c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.933 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:33.933 7fc6dafe6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.933 7fc6dafe6c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:33.933 7fc6dafe6c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.934 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:37:33.934 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:37:33.935 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:37:33.935 7fc6dafe6f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:33.936 7fc6daf14220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.936 7fc6daf14220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:33.936 7fc6daf14220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.936 7fc6daf14220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.936 7fc6daf14220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:33.936 7fc6daf14220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.940 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:37:33.942 7fc6dafe6f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:33.942 7fc6daf149a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.942 7fc6daf149a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:33.942 7fc6daf149a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.965 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:33.978 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:33.978 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:33.980 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:33.980 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:33.981 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:33.981 7fc6db09d7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:37:33.984 7fc6dc734400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:33.984 7fc6dc734400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:33.984 7fc6dc734400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:34.087 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:34.088 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:34.088 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:34.088 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:34.089 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:34.089 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:34.341 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:34.341 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:34.342 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:34.342 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:34.342 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:34.343 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:34.770 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:34.873 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:34.874 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:34.874 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:34.875 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:34.875 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:34.875 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:35.772 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:36.310 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:36.311 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:36.311 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:36.312 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:36.312 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:36.313 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:36.773 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:37.765 7fc6dc734400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:37.766 7fc6dc734400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:37.766 7fc6dc734400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:37.766 7fc6dc734400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:37.766 7fc6dc734400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:37.766 7fc6dc734400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:37.766 7fc6dc734400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:37.766 7fc6dc734400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:37:37.767 7fc6dc734400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:37.767 7fc6dc734400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:37.767 7fc6dc734400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:37.767 7fc6dc734900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:37:37.768 7fc6dc734900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:37:37.774 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:38.147 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:38.148 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:38.148 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:38.149 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:38.149 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:38.149 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:38.775 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:39.777 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:40.778 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:41.779 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:42.509 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:42.509 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:42.510 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:42.511 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:42.511 7fc6db04cc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:42.512 7fc6db04cc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:42.780 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:43.781 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:44.784 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:45.785 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:46.785 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:47.787 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:48.788 7fc6db09df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:48.840 7fc6db04cc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:37:48.841 7fc6db04cc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:37:48.841 7fc6db04cc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:49.058 7fc6db09d7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:37:49.059 7fc6dfe40600 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:37:49.059 7fc6dfe40600 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:37:49.059 7fc6db04c680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:37:49.059 7fc6db04c680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:37:49.079 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:37:49.080 7fc6dfe40600 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:37:58.015 7f2918a05540 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:37:58.015 7f2918a05540 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:37:58.015 7f2918a05540 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:37:58.018 7f2918a05540 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:37:58.019 7f2913f792c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:37:58.019 7f2913f792c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:37:58.019 7f2913f797c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:37:58.020 7f2913f797c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:37:58.020 7f2913f797c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:37:58.020 7f2913f797c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:37:58.020 7f2913f797c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.020 7f2913f797c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:37:58.020 7f2913f797c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.021 7f2913f797c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.021 7f2913f797c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353478020333 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:37:58.021 7f2913f797c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.021 7f2913f797c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.021 7f2913f797c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:58.022 7f2913f797c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.022 7f2913f79e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:58.023 7f2913f79e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:37:58.024 7f2913f79c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:37:58.024 7f2913f29540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.025 7f2913f29540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:37:58.025 7f2913f29540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.025 7f2913f29540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.025 7f2913f29540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:58.025 7f2913f29540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.077 7f2913f797c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:37:58.084 7f2913f797c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:37:58.085 7f2913f0f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.085 7f2913f0f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:58.085 7f2913f0f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.085 7f2913f0f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.086 7f2913f0f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:58.086 7f2913f0f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.087 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:58.088 7f2913f297c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.088 7f2913f297c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:58.088 7f2913f297c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.089 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.089 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:58.090 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.090 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:58.090 7f2913f797c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:37:58.091 7f2913f297c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.091 7f2913f297c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:58.091 7f2913f297c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.091 7f2913f297c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.092 7f2913f297c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:58.092 7f2913f297c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.092 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.099 7f2913f0fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.099 7f2913f0fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:58.099 7f2913f0fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.177 7f2913f797c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:58.177 7f2913f0fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.178 7f2913f0fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:37:58.178 7f2913f0fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.181 7f2913f797c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:58.182 7f2913f0fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.182 7f2913f0fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:37:58.182 7f2913f0fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.182 7f2913f797c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:37:58.182 7f2913f797c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:37:58.183 7f2913f797c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:37:58.183 7f2913f0fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:37:58.184 7f29138470e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.184 7f29138470e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:58.184 7f29138470e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.184 7f29138470e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.184 7f29138470e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:58.184 7f29138470e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.188 7f2913f797c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:37:58.190 7f2913f0fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:37:58.190 7f29138479a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.190 7f29138479a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:37:58.190 7f29138479a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.213 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:58.215 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.216 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:58.216 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.216 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:58.217 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.219 7f2913f797c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:37:58.221 7f29138477c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.221 7f29138477c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:37:58.222 7f29138477c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:58.482 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:58.483 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.483 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:58.484 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.484 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:58.484 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.845 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:58.846 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.846 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:58.847 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:58.847 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:58.847 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:59.023 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:37:59.416 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:37:59.416 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:59.416 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:37:59.417 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:37:59.417 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:37:59.418 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:00.025 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:00.346 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:00.346 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:00.347 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:00.347 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:00.347 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:00.348 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:01.027 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:02.020 7f29138477c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:02.021 7f29138477c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:02.021 7f29138477c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:02.021 7f29138477c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:02.022 7f29138477c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:02.022 7f29138477c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:02.022 7f29138477c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:02.022 7f29138477c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:38:02.022 7f29138477c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:02.022 7f29138477c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:02.022 7f29138477c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:02.022 7f291388e2c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:38:02.023 7f291388e2c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:38:02.028 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:02.378 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:02.379 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:02.379 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:02.380 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:02.380 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:02.381 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:03.029 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:04.031 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:05.032 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:06.033 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:07.035 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:07.559 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:07.560 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:07.560 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:07.561 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:07.561 7f2913f29a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:07.561 7f2913f29a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:08.036 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:09.037 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:10.038 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:11.040 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:12.041 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:13.042 7f2913f79f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:13.088 7f2913f29a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:38:13.088 7f2913f29a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:38:13.089 7f2913f29a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:13.234 7f2913f797c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:38:13.234 7f2918a05540 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:38:13.234 7f2918a05540 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:38:13.235 7f2913f29860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:38:13.235 7f2913f29860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:38:13.251 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:38:13.252 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:38:13.252 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:38:13.252 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:38:13.252 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:38:13.252 7f2918a05540 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:38:22.328 7f70dda3cd00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:38:22.328 7f70dda3cd00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:38:22.328 7f70dda3cd00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:38:22.332 7f70dda3cd00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:38:22.332 7f70d8c6b2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:38:22.332 7f70d8c6b2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:38:22.333 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:38:22.333 7f70d8c6b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:22.333 7f70d8c6b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:38:22.334 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:38:22.334 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.334 7f70d8c6b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:38:22.334 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.334 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.334 7f70d8c6b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353502333864 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:38:22.334 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.335 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.335 7f70d8c6b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:22.335 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.335 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.335 7f70d8c6b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:22.335 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.335 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.336 7f70d8c6b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:22.336 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.336 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.336 7f70d8c6b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:22.336 7f70d8c6b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.336 7f70d8c6be00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:22.336 7f70d8c6be00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:38:22.337 7f70d8c6bc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:38:22.338 7f70d8c1b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.338 7f70d8c1b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:22.338 7f70d8c1b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.338 7f70d8c1b540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.338 7f70d8c1b540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:22.338 7f70d8c1b540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.397 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:38:22.406 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:38:22.407 7f70d8bb8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.407 7f70d8bb8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:22.407 7f70d8bb8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.407 7f70d8bb8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.407 7f70d8bb8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:22.407 7f70d8bb8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.409 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:22.410 7f70d8c1b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.410 7f70d8c1b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:22.410 7f70d8c1b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.411 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.411 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:22.412 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:38:22.413 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.413 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:22.413 7f70d8c1b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.413 7f70d8c1b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:22.413 7f70d8c1b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.414 7f70d8bb8040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.415 7f70d8bb8040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:22.415 7f70d8bb8040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.415 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.423 7f70d8bb8c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.423 7f70d8bb8c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:22.423 7f70d8bb8c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.455 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:22.456 7f70d8bb8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.456 7f70d8bb8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:22.456 7f70d8bb8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.460 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:22.460 7f70d8bb8ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.460 7f70d8bb8ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:22.460 7f70d8bb8ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.461 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:38:22.461 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:38:22.461 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:38:22.461 7f70d8bb8e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:22.462 7f70d8ae80e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.462 7f70d8ae80e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:22.462 7f70d8ae80e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.462 7f70d8ae80e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.462 7f70d8ae80e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:22.462 7f70d8ae80e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.466 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:38:22.469 7f70d8bb8e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:22.469 7f70d8ae8c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.469 7f70d8ae8c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:22.470 7f70d8ae8c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.490 7f70d8c6b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:38:22.492 7f70da2ea180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.492 7f70da2ea180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:22.493 7f70da2ea180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:22.502 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:22.503 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.503 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:22.503 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.503 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:22.504 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.663 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:22.664 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.665 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:22.665 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.665 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:22.665 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.877 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:22.878 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.878 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:22.879 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:22.879 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:22.879 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:23.150 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:23.151 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:23.151 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:23.151 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:23.151 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:23.152 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:23.338 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:23.687 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:23.688 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:23.688 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:23.689 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:23.689 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:23.690 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:24.339 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:24.792 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:24.792 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:24.793 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:24.793 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:24.793 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:24.793 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:25.340 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:26.334 7f70da2ea680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:26.335 7f70da2ea680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:26.335 7f70da2ea680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:26.335 7f70da2ea680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:26.336 7f70da2ea680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:26.336 7f70da2ea680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:26.336 7f70da2ea680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:26.336 7f70da2ea680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:38:26.336 7f70da2ea680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:26.336 7f70da2ea680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:26.336 7f70da2ea680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:26.337 7f70da2ea900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:38:26.338 7f70da2ea900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:38:26.341 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:26.485 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:26.486 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:26.486 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:26.487 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:26.487 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:26.487 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:27.342 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:28.344 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:29.346 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:30.347 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:30.917 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:30.918 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.918 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:30.919 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:30.919 7f70d8c1ba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:30.919 7f70d8c1ba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:31.348 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:32.349 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:33.350 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:34.352 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:35.352 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:36.353 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:37.348 7f70d8c1ba40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:38:37.348 7f70d8c1ba40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:38:37.349 7f70d8c1ba40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:37.354 7f70d8c6bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:37.545 7f70d8c6b7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:38:37.545 7f70dda3cd00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:38:37.546 7f70dda3cd00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:38:37.546 7f70d8c1b860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:38:37.546 7f70d8c1b860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:38:37.565 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:38:37.565 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:38:37.565 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:38:37.565 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:38:37.566 7f70dda3cd00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:38:46.733 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:38:46.733 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:38:46.733 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:38:46.737 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:38:46.737 7f9a5492c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:38:46.737 7f9a5492c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:38:46.738 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:38:46.738 7f9a5492c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:38:46.738 7f9a5492c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:38:46.739 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:38:46.739 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.739 7f9a5492c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:38:46.739 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.739 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.739 7f9a5492c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353526738862 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:38:46.739 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.740 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.740 7f9a5492c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:46.740 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.740 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.740 7f9a5492c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:46.740 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.741 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.741 7f9a5492c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:38:46.741 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.741 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.741 7f9a5492c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:46.741 7f9a5492c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.741 7f9a5492ce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:46.741 7f9a5492ce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:38:46.743 7f9a5492cc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:38:46.743 7f9a548dc540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.743 7f9a548dc540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:38:46.744 7f9a548dc540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.744 7f9a548dc540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.744 7f9a548dc540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:46.744 7f9a548dc540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.802 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:38:46.811 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:38:46.812 7f9a548740e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.812 7f9a548740e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:46.812 7f9a548740e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.812 7f9a548740e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.813 7f9a548740e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:46.813 7f9a548740e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.814 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:46.815 7f9a548dc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.816 7f9a548dc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:46.816 7f9a548dc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.816 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:46.816 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:46.818 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:38:46.818 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:46.818 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:46.818 7f9a548dc7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.818 7f9a548dc7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:46.818 7f9a548dc7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.819 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:46.827 7f9a54874d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.827 7f9a54874d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:46.827 7f9a54874d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.829 7f9a54874d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.830 7f9a54874d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:46.830 7f9a54874d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.898 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:46.899 7f9a54874c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.899 7f9a54874c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:38:46.899 7f9a54874c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.902 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:46.903 7f9a54874c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.903 7f9a54874c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:38:46.903 7f9a54874c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.903 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:38:46.904 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:38:46.904 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:38:46.904 7f9a54874f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:38:46.905 7f9a547b0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.905 7f9a547b0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:46.905 7f9a547b0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.905 7f9a547b0220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.905 7f9a547b0220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:46.905 7f9a547b0220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.909 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:38:46.911 7f9a54874f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:38:46.911 7f9a547b09a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.911 7f9a547b09a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:38:46.911 7f9a547b09a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.918 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:46.919 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:46.919 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:46.920 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:46.920 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:46.923 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:46.937 7f9a5492c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:38:46.940 7f9a55f02400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:46.940 7f9a55f02400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:46.940 7f9a55f02400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:47.118 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:47.120 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:47.120 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:47.120 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:47.120 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:47.121 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:47.356 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:47.357 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:47.357 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:47.357 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:47.357 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:47.358 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:47.743 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:47.959 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:47.960 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:47.960 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:47.961 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:47.961 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:47.961 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:48.744 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:49.506 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:49.507 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:49.507 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:49.507 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:49.507 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:49.508 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:49.746 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:50.738 7f9a55f02400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:50.740 7f9a55f02400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:50.740 7f9a55f02400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:50.740 7f9a55f02400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:50.740 7f9a55f02400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:38:50.740 7f9a55f02400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:50.740 7f9a55f02400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:50.741 7f9a55f02400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:38:50.741 7f9a55f02400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:50.741 7f9a55f02400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:50.741 7f9a55f02400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:50.741 7f9a55f02900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:38:50.743 7f9a55f02900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:38:50.747 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:51.639 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:51.640 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:51.640 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:51.640 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:51.640 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:51.641 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:51.748 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:52.749 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:53.751 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:54.752 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:55.754 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:56.755 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:57.539 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:38:57.540 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:57.540 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:38:57.540 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:57.540 7f9a548dcc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:38:57.541 7f9a548dcc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:38:57.757 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:58.759 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:38:59.759 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:00.760 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:01.760 7f9a548dcc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:39:01.760 7f9a548dcc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:39:01.761 7f9a548dcc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:01.762 7f9a5492cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:02.020 7f9a5492c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:39:02.020 7f9a593bbbc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:39:02.020 7f9a593bbbc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:39:02.020 7f9a548dc680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:39:02.020 7f9a548dc680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:39:02.045 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:39:02.045 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:39:02.045 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:39:02.045 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:39:02.045 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:39:02.045 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:39:02.045 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:39:02.045 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:39:02.045 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:39:02.046 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:39:02.046 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:39:02.046 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:39:02.046 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:39:02.046 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:39:02.046 7f9a593bbbc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:39:11.422 7f40d5e2d340 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:39:11.423 7f40d5e2d340 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:39:11.423 7f40d5e2d340 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:39:11.427 7f40d5e2d340 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:39:11.428 7f40d0ba62c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:39:11.428 7f40d0ba62c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:39:11.429 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:39:11.430 7f40d0ba67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:39:11.430 7f40d0ba67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:39:11.430 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:39:11.430 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.430 7f40d0ba67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:39:11.430 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.431 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.431 7f40d0ba67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353551430357 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:39:11.431 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.433 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.434 7f40d0ba67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:11.434 7f40d0ba67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.435 7f40d0ba6e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:11.435 7f40d0ba6e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:39:11.437 7f40d0ba6c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:39:11.439 7f40d0b56540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.439 7f40d0b56540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:11.439 7f40d0b56540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.439 7f40d0b56540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.440 7f40d0b56540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:11.440 7f40d0b56540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.540 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:39:11.552 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:39:11.554 7f40d2cb80e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.554 7f40d2cb80e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:11.554 7f40d2cb80e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.554 7f40d2cb80e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.554 7f40d2cb80e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:11.555 7f40d2cb80e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.556 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:11.557 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:11.557 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:11.559 7f40d0b567c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.559 7f40d0b567c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:11.559 7f40d0b567c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.560 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:11.560 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:11.561 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:11.563 7f40d0b567c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.563 7f40d0b567c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:11.563 7f40d0b567c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.564 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:39:11.565 7f40d0b567c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.565 7f40d0b567c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:11.565 7f40d0b567c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.575 7f40d2cb8d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.575 7f40d2cb8d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:11.575 7f40d2cb8d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.600 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:11.601 7f40d2cb8c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.601 7f40d2cb8c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:11.601 7f40d2cb8c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.605 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:11.606 7f40d2cb8c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.606 7f40d2cb8c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:11.606 7f40d2cb8c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.606 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:39:11.606 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:39:11.607 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:39:11.607 7f40d2cb8f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:11.608 7f40d0b38220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.608 7f40d0b38220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:11.608 7f40d0b38220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.608 7f40d0b38220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.608 7f40d0b38220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:11.608 7f40d0b38220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.611 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:39:11.615 7f40d2cb8f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:11.616 7f40d0b38c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.616 7f40d0b38c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:11.616 7f40d0b38c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.637 7f40d0ba67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:39:11.638 7f40d216d400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.639 7f40d216d400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:11.639 7f40d216d400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:11.700 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:11.701 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:11.701 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:11.702 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:11.702 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:11.702 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:12.003 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:12.004 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:12.004 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:12.005 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:12.005 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:12.006 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:12.436 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:12.693 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:12.694 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:12.694 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:12.695 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:12.695 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:12.696 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:13.437 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:13.686 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:13.686 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:13.687 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:13.687 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:13.687 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:13.688 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:14.439 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:15.431 7f40d216d400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:15.432 7f40d216d400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:15.432 7f40d216d400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:15.432 7f40d216d400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:15.432 7f40d216d400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:15.432 7f40d216d400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:15.432 7f40d216d400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:15.432 7f40d216d400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:39:15.433 7f40d216d400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:15.433 7f40d216d400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:15.433 7f40d216d400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:15.433 7f40d216d900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:39:15.434 7f40d216d900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:39:15.440 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:16.001 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:16.002 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:16.002 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:16.003 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:16.003 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:16.003 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:16.441 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:17.442 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:18.443 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:19.444 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:19.796 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:19.797 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.797 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:19.797 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:19.798 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:19.798 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:20.445 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:21.446 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:22.447 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:23.448 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:24.444 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:24.444 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:24.444 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:24.445 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:24.445 7f40d0b56c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:24.445 7f40d0b56c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:24.450 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:25.451 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:26.452 7f40d0ba6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:26.455 7f40d0b56c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:39:26.455 7f40d0b56c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:39:26.456 7f40d0b56c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:26.651 7f40d0ba67c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:39:26.651 7f40d5e2d340 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:39:26.651 7f40d5e2d340 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:39:26.652 7f40d0b56680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:39:26.652 7f40d0b56680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:39:26.665 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:39:26.665 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:39:26.665 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:39:26.666 7f40d5e2d340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:39:35.895 7f657543c640 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:39:35.895 7f657543c640 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:39:35.895 7f657543c640 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:39:35.899 7f657543c640 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:39:35.899 7f65709b82c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:39:35.899 7f65709b82c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:39:35.900 7f65709b87c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:39:35.900 7f65709b87c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:39:35.900 7f65709b87c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:39:35.901 7f65709b87c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:39:35.901 7f65709b87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.901 7f65709b87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:39:35.901 7f65709b87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.901 7f65709b87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.901 7f65709b87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353575900916 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:39:35.901 7f65709b87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.902 7f65709b87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.902 7f65709b87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:35.902 7f65709b87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.903 7f65709b87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.903 7f65709b87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:35.903 7f65709b87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.903 7f65709b87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.903 7f65709b87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:39:35.903 7f65709b87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.903 7f65709b87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.903 7f65709b87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:35.903 7f65709b87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.904 7f65709b8e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:35.904 7f65709b8e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:39:35.905 7f65709b8c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:39:35.906 7f6570968540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.906 7f6570968540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:39:35.906 7f6570968540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.906 7f6570968540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.907 7f6570968540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:35.907 7f6570968540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:35.991 7f65709b87c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:39:36.001 7f65709b87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:39:36.002 7f6572ac90e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.002 7f6572ac90e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:36.002 7f6572ac90e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.002 7f6572ac90e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.003 7f6572ac90e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:36.003 7f6572ac90e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.004 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:36.005 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.005 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:36.006 7f65709687c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.007 7f65709687c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:36.007 7f65709687c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.008 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.008 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:36.008 7f65709b87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:39:36.009 7f65709687c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.009 7f65709687c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:36.009 7f65709687c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.009 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.011 7f6572ac90e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.011 7f6572ac90e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:36.011 7f6572ac90e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.020 7f6572ac9d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.021 7f6572ac9d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:36.021 7f6572ac9d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.053 7f65709b87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:36.054 7f6572ac9c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.055 7f6572ac9c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:39:36.055 7f6572ac9c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.058 7f65709b87c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:36.059 7f6572ac9c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.059 7f6572ac9c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:39:36.059 7f6572ac9c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.060 7f65709b87c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:39:36.060 7f65709b87c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:39:36.060 7f65709b87c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:39:36.060 7f6572ac9f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:39:36.061 7f6571f90220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.061 7f6571f90220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:36.062 7f6571f90220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.062 7f6571f90220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.062 7f6571f90220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:36.062 7f6571f90220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.066 7f65709b87c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:39:36.074 7f6572ac9f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:39:36.074 7f6571f90ea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.075 7f6571f90ea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:39:36.075 7f6571f90ea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.076 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:36.077 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.078 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:36.100 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.101 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:36.104 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.104 7f65709b87c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:39:36.107 7f6571f782c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.107 7f6571f782c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:36.107 7f6571f782c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:36.200 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:36.201 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.202 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:36.202 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.202 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:36.203 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.405 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:36.406 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.406 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:36.407 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.407 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:36.407 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.880 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:36.881 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.881 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:36.882 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.882 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:36.883 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:36.905 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:37.877 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:37.878 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:37.878 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:37.878 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:37.878 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:37.879 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:37.906 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:38.908 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:39.900 7f6571f907c0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:39.902 7f6571f907c0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:39.902 7f6571f907c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:39.902 7f6571f907c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:39.902 7f6571f907c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:39:39.902 7f6571f907c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:39.902 7f6571f907c0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:39.902 7f6571f907c0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:39:39.903 7f6571f907c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:39.903 7f6571f907c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:39.903 7f6571f907c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:39.903 7f6571f787c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:39:39.904 7f6571f787c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:39:39.909 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:40.471 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:40.472 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:40.472 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:40.473 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:40.473 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:40.473 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:40.910 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:41.912 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:42.913 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:43.914 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:44.914 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:45.915 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:46.917 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:46.964 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:39:46.969 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:46.969 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:39:46.970 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:46.970 7f6570968c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:39:46.971 7f6570968c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:39:47.918 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:48.919 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:49.920 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:50.921 7f65709b8f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:39:50.991 7f6570968c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:39:50.991 7f6570968c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:39:50.992 7f6570968c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:51.201 7f65709b87c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:39:51.202 7f657543c640 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:39:51.202 7f657543c640 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:39:51.202 7f6570968b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:39:51.202 7f6570968b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:39:51.217 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:39:51.217 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:39:51.217 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:39:51.217 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:39:51.217 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:39:51.217 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:39:51.218 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:39:51.218 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:39:51.218 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:39:51.218 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:39:51.218 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:39:51.218 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:39:51.218 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:39:51.218 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:39:51.218 7f657543c640 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:40:00.678 7fb08e718e80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:40:00.678 7fb08e718e80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:40:00.678 7fb08e718e80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:40:00.682 7fb08e718e80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:40:00.682 7fb0899502c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:40:00.683 7fb0899502c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:40:00.683 7fb0899507c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:40:00.684 7fb0899507c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:00.684 7fb0899507c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:40:00.684 7fb0899507c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:40:00.684 7fb0899507c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.684 7fb0899507c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:40:00.684 7fb0899507c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.684 7fb0899507c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.684 7fb0899507c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353600684273 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:40:00.685 7fb0899507c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.685 7fb0899507c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.685 7fb0899507c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:00.685 7fb0899507c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.686 7fb0899507c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.686 7fb0899507c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:00.686 7fb0899507c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.686 7fb0899507c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.686 7fb0899507c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:00.686 7fb0899507c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.686 7fb0899507c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.686 7fb0899507c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:00.686 7fb0899507c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.686 7fb089950e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:00.687 7fb089950e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:40:00.688 7fb089950c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:40:00.688 7fb0898ff540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.688 7fb0898ff540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:00.688 7fb0898ff540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.689 7fb0898ff540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.689 7fb0898ff540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:00.689 7fb0898ff540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.751 7fb0899507c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:40:00.760 7fb0899507c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:40:00.761 7fb08989f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.762 7fb08989f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:00.762 7fb08989f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.762 7fb08989f040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.762 7fb08989f040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:00.762 7fb08989f040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.763 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:00.764 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:00.764 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:00.764 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:00.764 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:00.765 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:00.766 7fb0898ff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.766 7fb0898ff7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:00.766 7fb0898ff7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.768 7fb0899507c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:40:00.768 7fb0898ff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.768 7fb0898ff7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:00.768 7fb0898ff7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.769 7fb0898ff7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.769 7fb0898ff7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:00.769 7fb0898ff7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.777 7fb08989fc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.778 7fb08989fc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:00.778 7fb08989fc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.806 7fb0899507c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:00.807 7fb08989fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.808 7fb08989fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:00.808 7fb08989fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.811 7fb0899507c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:00.812 7fb08989fae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.812 7fb08989fae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:00.812 7fb08989fae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.812 7fb0899507c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:40:00.813 7fb0899507c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:40:00.813 7fb0899507c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:40:00.813 7fb08989fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:00.815 7fb0897d40e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.815 7fb0897d40e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:00.815 7fb0897d40e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.815 7fb0897d40e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.815 7fb0897d40e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:00.815 7fb0897d40e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.820 7fb0899507c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:40:00.823 7fb08989fe00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:00.823 7fb0897d4860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.824 7fb0897d4860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:00.824 7fb0897d4860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.866 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:00.871 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:00.871 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:00.873 7fb0899507c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:40:00.874 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:00.874 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:00.878 7fb08af0e180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.878 7fb08af0e180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:00.879 7fb08af0e180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:00.879 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.032 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:01.033 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.033 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:01.034 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.034 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:01.034 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.322 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:01.323 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.323 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:01.323 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.324 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:01.324 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.688 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:01.850 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:01.851 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.851 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:01.851 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:01.851 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:01.852 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:02.689 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:02.862 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:02.862 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:02.863 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:02.863 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:02.863 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:02.864 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:03.690 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:04.683 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:04.684 7fb08af0e680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:04.685 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:04.685 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:04.685 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:04.685 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:04.686 7fb08af0e680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:04.686 7fb08af0e680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:04.686 7fb08af0e680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:04.686 7fb08af0e680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:04.686 7fb08af0e680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:04.686 7fb08af0e680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:04.686 7fb08af0e680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:40:04.687 7fb08af0e680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:04.687 7fb08af0e680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:04.687 7fb08af0e680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:04.687 7fb08af0e900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:40:04.689 7fb08af0e900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:40:04.689 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:04.692 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:05.693 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:06.694 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:07.655 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:07.656 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.656 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:07.656 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.656 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:07.657 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:07.695 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:08.696 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:09.698 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:10.699 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:11.700 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:12.702 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:12.730 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:12.731 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:12.731 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:12.732 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:12.732 7fb0898ffa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:12.733 7fb0898ffa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:13.703 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:14.705 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:15.706 7fb089950f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:15.747 7fb0898ffa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:40:15.747 7fb0898ffa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:40:15.748 7fb0898ffa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:15.885 7fb0899507c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:40:15.885 7fb08e718e80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:40:15.885 7fb08e718e80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:40:15.885 7fb0898ff680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:40:15.885 7fb0898ff680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:40:15.899 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:40:15.900 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:40:15.900 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:40:15.900 7fb08e718e80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:40:25.071 7f8f3f03c340 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:40:25.072 7f8f3f03c340 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:40:25.072 7f8f3f03c340 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:40:25.075 7f8f3f03c340 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:40:25.076 7f8f39fca2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:40:25.077 7f8f39fca2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:40:25.077 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:40:25.077 7f8f39fca7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:25.078 7f8f39fca7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:40:25.078 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:40:25.078 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.078 7f8f39fca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:40:25.078 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.078 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.078 7f8f39fca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353625077982 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:40:25.078 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.079 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.080 7f8f39fca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:25.080 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.080 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.080 7f8f39fca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:25.080 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.080 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.080 7f8f39fca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:25.080 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.081 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.081 7f8f39fca7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.081 7f8f39fca7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.081 7f8f39fcae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:25.081 7f8f39fcae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:40:25.085 7f8f39fcac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:40:25.085 7f8f39f7a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.086 7f8f39f7a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:25.086 7f8f39f7a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.086 7f8f39f7a540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.086 7f8f39f7a540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.086 7f8f39f7a540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.169 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:40:25.183 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:40:25.185 7f8f39ed9040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.185 7f8f39ed9040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:25.185 7f8f39ed9040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.186 7f8f39ed9040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.186 7f8f39ed9040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:25.186 7f8f39ed9040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.187 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:25.188 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.188 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:25.190 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.190 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:25.191 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:40:25.191 7f8f39f7a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.192 7f8f39f7a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.192 7f8f39f7a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.192 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.193 7f8f39ed9040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.194 7f8f39ed9040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.194 7f8f39ed9040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.207 7f8f39ed9c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.208 7f8f39ed9c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.208 7f8f39ed9c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.209 7f8f39ed9c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.209 7f8f39ed9c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.210 7f8f39ed9c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.238 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:25.239 7f8f39ed9ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.240 7f8f39ed9ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:25.240 7f8f39ed9ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.242 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:25.243 7f8f39ed9ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.243 7f8f39ed9ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:25.243 7f8f39ed9ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.245 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:40:25.245 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:40:25.245 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:40:25.246 7f8f39ed9e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:25.246 7f8f39e4c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.247 7f8f39e4c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:25.247 7f8f39e4c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.247 7f8f39e4c0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.247 7f8f39e4c0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:25.247 7f8f39e4c0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.251 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:40:25.255 7f8f39ed9e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:25.256 7f8f39e4cc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.256 7f8f39e4cc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:25.256 7f8f39e4cc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.288 7f8f39fca7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:40:25.291 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.292 7f8f39e4c900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:25.293 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:25.304 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:25.306 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.306 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:25.307 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.307 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:25.307 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.473 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:25.474 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.474 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:25.475 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.475 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:25.476 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.756 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:25.757 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.758 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:25.758 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:25.759 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:25.759 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:26.085 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:26.500 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:26.501 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:26.501 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:26.501 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:26.501 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:26.502 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:27.086 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:27.955 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:27.956 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:27.956 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:27.956 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:27.957 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:27.957 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:28.088 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:29.078 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:29.079 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:29.080 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:29.080 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:29.080 7f8f39e4c900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:29.080 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:29.080 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:29.080 7f8f39e4c900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:40:29.081 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:29.081 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:29.081 7f8f39e4c900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:29.081 7f8f3b594180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:40:29.082 7f8f3b594180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:40:29.088 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:30.090 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:30.132 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:30.133 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:30.133 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:30.133 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:30.133 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:30.134 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:31.091 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:32.092 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:33.093 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:34.094 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:34.235 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:34.236 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:34.236 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:34.236 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:34.236 7f8f39f7aa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:34.237 7f8f39f7aa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:35.096 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:36.097 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:37.099 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:38.100 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:39.101 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:40.102 7f8f39fcaf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:40.164 7f8f39f7aa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:40:40.164 7f8f39f7aa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:40:40.165 7f8f39f7aa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:40.323 7f8f39fca7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:40:40.323 7f8f3f03c340 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:40:40.324 7f8f3f03c340 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:40:40.324 7f8f39f7a860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:40:40.324 7f8f39f7a860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:40:40.341 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:40:40.342 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:40:40.342 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:40:40.342 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:40:40.342 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:40:40.342 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:40:40.342 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:40:40.342 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:40:40.342 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:40:40.342 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:40:40.343 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:40:40.343 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:40:40.343 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:40:40.343 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:40:40.343 7f8f3f03c340 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:40:49.868 7fc847988c40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:40:49.868 7fc847988c40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:40:49.868 7fc847988c40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:40:49.876 7fc847988c40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:40:49.877 7fc84292f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:40:49.877 7fc84292f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:40:49.877 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:40:49.879 7fc84292f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:40:49.879 7fc84292f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:40:49.879 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:40:49.879 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.879 7fc84292f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:40:49.879 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.879 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.881 7fc84292f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353649879401 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:40:49.881 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.883 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.883 7fc84292f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:49.884 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.884 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.884 7fc84292f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:49.884 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.885 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.885 7fc84292f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:40:49.885 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.885 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.885 7fc84292f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:49.885 7fc84292f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.886 7fc84292fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:49.887 7fc84292fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:40:49.888 7fc84292fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:40:49.889 7fc8428df540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.890 7fc8428df540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:40:49.890 7fc8428df540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.890 7fc8428df540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:49.891 7fc8428df540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:49.891 7fc8428df540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.032 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:40:50.054 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:40:50.055 7fc842872040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.055 7fc842872040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:50.056 7fc842872040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.057 7fc842872040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.057 7fc842872040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:50.057 7fc842872040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.058 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:50.058 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.058 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:50.061 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.061 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:50.065 7fc8428df7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.066 7fc8428df7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:50.066 7fc8428df7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.066 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.066 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:40:50.067 7fc8428df7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.068 7fc8428df7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:50.068 7fc8428df7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.070 7fc842872040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.072 7fc842872040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:50.072 7fc842872040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.085 7fc842872c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.085 7fc842872c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:50.085 7fc842872c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.108 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:50.109 7fc842872ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.110 7fc842872ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:40:50.110 7fc842872ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.113 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:50.114 7fc842872ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.114 7fc842872ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:40:50.114 7fc842872ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.114 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:40:50.115 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:40:50.115 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:40:50.115 7fc842872e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:40:50.116 7fc84283e0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.116 7fc84283e0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:50.116 7fc84283e0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.116 7fc84283e0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.116 7fc84283e0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:50.116 7fc84283e0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.120 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:40:50.125 7fc842872e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:40:50.125 7fc84283e860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.126 7fc84283e860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:40:50.126 7fc84283e860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.203 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:50.207 7fc84292f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:40:50.208 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.209 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:50.211 7fc8427bf400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.212 7fc8427bf400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:50.212 7fc8427bf400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:50.217 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.217 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:50.219 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.476 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:50.477 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.477 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:50.477 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.478 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:50.478 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.888 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:50.889 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:50.890 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.890 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:50.890 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:50.891 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:50.891 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:51.871 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:51.872 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:51.872 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:51.873 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:51.873 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:51.873 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:51.889 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:52.890 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:53.878 7fc8427bf400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:53.880 7fc8427bf400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:53.880 7fc8427bf400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:53.880 7fc8427bf400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:53.880 7fc8427bf400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:40:53.880 7fc8427bf400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:53.880 7fc8427bf400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:53.881 7fc8427bf400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:40:53.881 7fc8427bf400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:53.881 7fc8427bf400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:53.881 7fc8427bf400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:53.882 7fc8427bf900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:40:53.883 7fc8427bf900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:40:53.892 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:54.305 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:54.306 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:54.307 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:54.307 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:54.307 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:54.308 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:54.893 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:55.894 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:56.895 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:57.531 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:40:57.532 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:57.533 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:40:57.533 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:57.533 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:40:57.534 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:40:57.897 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:58.898 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:40:59.899 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:00.900 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:01.900 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:02.901 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:03.902 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:04.284 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:04.285 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:04.285 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:04.286 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:04.286 7fc8428dfa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:04.287 7fc8428dfa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:04.904 7fc84292ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:04.990 7fc8428dfa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:41:04.990 7fc8428dfa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:41:04.991 7fc8428dfa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:05.257 7fc84292f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:41:05.258 7fc847988c40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:41:05.258 7fc847988c40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:41:05.258 7fc8428df860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:41:05.258 7fc8428df860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:41:05.272 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:41:05.273 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:41:05.273 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:41:05.273 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:41:05.273 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:41:05.273 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:41:05.273 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:41:05.273 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:41:05.273 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:41:05.273 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:41:05.274 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:41:05.274 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:41:05.274 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:41:05.274 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:41:05.274 7fc847988c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:41:14.596 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:41:14.596 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:41:14.596 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:41:14.601 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:41:14.602 7f75aa6482c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:41:14.602 7f75aa6482c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:41:14.602 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:41:14.603 7f75aa6487c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:41:14.603 7f75aa6487c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:41:14.604 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:41:14.604 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.604 7f75aa6487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:41:14.604 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.604 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.604 7f75aa6487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353674603738 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:41:14.604 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.605 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.606 7f75aa6487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:14.606 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.606 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.606 7f75aa6487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:14.606 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.606 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.607 7f75aa6487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:14.607 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.607 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.607 7f75aa6487c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:14.607 7f75aa6487c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.607 7f75aa648e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:14.608 7f75aa648e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:41:14.609 7f75aa648c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:41:14.610 7f75aa5f8540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.610 7f75aa5f8540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:14.610 7f75aa5f8540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.610 7f75aa5f8540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.610 7f75aa5f8540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:14.610 7f75aa5f8540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.739 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:41:14.755 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:41:14.757 7f75ac759040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.757 7f75ac759040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:14.757 7f75ac759040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.757 7f75ac759040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.758 7f75ac759040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:14.758 7f75ac759040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.760 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:14.761 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:14.761 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:14.763 7f75aa5f87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.764 7f75aa5f87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:14.764 7f75aa5f87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.765 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:14.765 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:14.768 7f75aa5f87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.769 7f75aa5f87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:14.769 7f75aa5f87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.769 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:14.769 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:41:14.770 7f75aa5f87c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.770 7f75aa5f87c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:14.771 7f75aa5f87c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.783 7f75ac759c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.783 7f75ac759c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:14.783 7f75ac759c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.839 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:14.840 7f75ac759ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.840 7f75ac759ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:14.840 7f75ac759ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.841 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:14.851 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:14.851 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:14.852 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:14.852 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:14.853 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:14.854 7f75ac759ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.854 7f75ac759ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:14.854 7f75ac759ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.854 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:14.855 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:41:14.855 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:41:14.855 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:41:14.856 7f75ac759e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:14.856 7f75aa5d00e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.857 7f75aa5d00e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:14.857 7f75aa5d00e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.857 7f75aa5d00e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.857 7f75aa5d00e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:14.857 7f75aa5d00e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.861 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:41:14.864 7f75ac759e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:14.865 7f75aa5d05e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.865 7f75aa5d05e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:14.865 7f75aa5d05e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.915 7f75aa6487c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:41:14.918 7f75abbf9400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.919 7f75abbf9400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:14.919 7f75abbf9400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:14.944 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:14.945 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:14.945 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:14.945 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:14.945 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:14.946 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:15.135 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:15.136 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:15.136 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:15.136 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:15.136 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:15.137 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:15.519 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:15.519 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:15.519 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:15.520 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:15.520 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:15.520 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:15.609 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:16.413 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:16.414 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:16.414 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:16.414 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:16.415 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:16.415 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:16.610 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:17.590 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:17.591 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:17.591 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:17.592 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:17.592 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:17.593 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:17.611 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:18.604 7f75abbf9400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:18.605 7f75abbf9400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:18.605 7f75abbf9400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:18.606 7f75abbf9400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:18.606 7f75abbf9400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:18.606 7f75abbf9400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:18.606 7f75abbf9400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:18.606 7f75abbf9400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:41:18.607 7f75abbf9400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:18.607 7f75abbf9400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:18.607 7f75abbf9400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:18.607 7f75abbf9900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:41:18.609 7f75abbf9900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:41:18.612 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:19.199 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:19.200 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:19.200 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:19.200 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:19.200 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:19.201 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:19.613 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:20.614 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:21.615 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:22.616 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:23.574 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:23.575 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:23.575 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:23.575 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:23.575 7f75aa5f8a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:23.576 7f75aa5f8a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:23.617 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:24.619 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:25.621 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:26.621 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:27.623 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:28.624 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:29.625 7f75aa648f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:29.705 7f75aa5f8a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:41:29.706 7f75aa5f8a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:41:29.707 7f75aa5f8a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:30.012 7f75aa6487c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:41:30.013 7f75af2a6ac0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:41:30.013 7f75af2a6ac0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:41:30.013 7f75aa5f8860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:41:30.013 7f75aa5f8860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:41:30.037 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:41:30.038 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:41:30.038 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:41:30.038 7f75af2a6ac0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:41:39.666 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:41:39.666 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:41:39.666 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:41:39.669 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:41:39.670 7f61ad11d2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:41:39.670 7f61ad11d2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:41:39.670 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:41:39.671 7f61ad11d7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:41:39.671 7f61ad11d7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:41:39.671 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:41:39.671 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.672 7f61ad11d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:41:39.672 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.672 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.672 7f61ad11d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353699671533 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:41:39.672 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:39.673 7f61ad11d7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.674 7f61ad11de00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:39.674 7f61ad11de00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:41:39.675 7f61ad11dc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:41:39.676 7f61ad0cd540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.676 7f61ad0cd540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:41:39.676 7f61ad0cd540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.676 7f61ad0cd540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.676 7f61ad0cd540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:39.676 7f61ad0cd540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.739 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:41:39.747 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:41:39.748 7f61ad0610e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.748 7f61ad0610e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:39.748 7f61ad0610e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.748 7f61ad0610e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.748 7f61ad0610e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:39.749 7f61ad0610e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.749 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:39.750 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:39.750 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:39.751 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:39.751 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:39.751 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:39.754 7f61ad0cd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.754 7f61ad0cd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:39.754 7f61ad0cd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.756 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:41:39.757 7f61ad0cd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.757 7f61ad0cd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:39.757 7f61ad0cd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.758 7f61ad0cd7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.758 7f61ad0cd7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:39.758 7f61ad0cd7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.767 7f61ad061d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.767 7f61ad061d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:39.767 7f61ad061d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.797 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:39.798 7f61ad061c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.799 7f61ad061c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:41:39.799 7f61ad061c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.802 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:39.803 7f61ad061c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.803 7f61ad061c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:41:39.803 7f61ad061c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.803 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:41:39.803 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:41:39.803 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:41:39.804 7f61ad061f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:41:39.804 7f61acf9f220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.805 7f61acf9f220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:39.805 7f61acf9f220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.805 7f61acf9f220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.805 7f61acf9f220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:39.805 7f61acf9f220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.809 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:41:39.811 7f61ad061f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:41:39.811 7f61acf9f9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.811 7f61acf9f9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:41:39.811 7f61acf9f9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.816 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:39.818 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:39.818 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:39.819 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:39.819 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:39.825 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:39.848 7f61ad11d7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:41:39.855 7f61acf9f900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.855 7f61acf9f900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:39.856 7f61acf9f900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:39.912 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:39.913 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:39.913 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:39.913 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:39.914 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:39.914 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.070 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:40.071 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.071 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:40.071 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.071 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:40.072 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.381 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:40.382 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.382 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:40.382 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.383 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:40.383 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.675 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:40.830 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:40.831 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.831 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:40.831 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:40.832 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:40.832 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:41.676 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:41.844 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:41.844 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:41.845 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:41.845 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:41.845 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:41.846 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:42.677 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:43.520 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:43.521 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:43.521 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:43.521 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:43.522 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:43.522 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:43.671 7f61acf9f900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:43.672 7f61acf9f900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:43.673 7f61acf9f900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:43.673 7f61acf9f900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:43.673 7f61acf9f900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:41:43.673 7f61acf9f900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:43.673 7f61acf9f900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:43.673 7f61acf9f900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:41:43.674 7f61acf9f900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:43.674 7f61acf9f900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:43.674 7f61acf9f900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:43.674 7f61af7e1180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:41:43.675 7f61af7e1180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:41:43.678 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:44.679 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:45.681 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:46.681 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:47.683 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:47.722 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:41:47.723 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.723 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:41:47.723 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:47.724 7f61ad0cdc20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:41:47.724 7f61ad0cdc20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:41:48.685 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:49.686 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:50.686 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:51.688 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:52.689 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:53.690 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:54.691 7f61ad11df40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:41:54.758 7f61ad0cdc20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:41:54.758 7f61ad0cdc20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:41:54.759 7f61ad0cdc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:54.870 7f61ad11d7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:41:54.871 7f61b1ee5e40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:41:54.871 7f61b1ee5e40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:41:54.871 7f61ad0cdb80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:41:54.871 7f61ad0cdb80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:41:54.888 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:41:54.888 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:41:54.888 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:41:54.888 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:41:54.888 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:41:54.888 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:41:54.888 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:41:54.889 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:41:54.889 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:41:54.889 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:41:54.889 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:41:54.889 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:41:54.889 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:41:54.889 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:41:54.889 7f61b1ee5e40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:42:04.126 7fda4f5179c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:42:04.126 7fda4f5179c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:42:04.126 7fda4f5179c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:42:04.132 7fda4f5179c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:42:04.133 7fda4a4c32c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:42:04.133 7fda4a4c32c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:42:04.133 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:42:04.134 7fda4a4c37c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:42:04.134 7fda4a4c37c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:42:04.135 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:42:04.135 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.135 7fda4a4c37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:42:04.135 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.135 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.135 7fda4a4c37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353724134862 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:42:04.135 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.136 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.136 7fda4a4c37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:04.136 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.137 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.137 7fda4a4c37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:04.137 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.137 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.137 7fda4a4c37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:04.137 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.137 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.137 7fda4a4c37c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:04.137 7fda4a4c37c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.137 7fda4a4c3e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:04.138 7fda4a4c3e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:42:04.139 7fda4a4c3c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:42:04.140 7fda4a472540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.140 7fda4a472540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:04.140 7fda4a472540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.140 7fda4a472540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.140 7fda4a472540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:04.140 7fda4a472540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.206 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:42:04.214 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:42:04.215 7fda4a3d6040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.215 7fda4a3d6040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:04.216 7fda4a3d6040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.216 7fda4a3d6040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.216 7fda4a3d6040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:04.216 7fda4a3d6040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.217 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:04.218 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:04.218 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:04.219 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:04.219 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:04.220 7fda4a4727c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.220 7fda4a4727c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:04.220 7fda4a4727c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.221 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:04.222 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:42:04.222 7fda4a4727c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.222 7fda4a4727c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:04.223 7fda4a4727c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.224 7fda4a3d6040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.224 7fda4a3d6040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:04.224 7fda4a3d6040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.232 7fda4a3d6c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.232 7fda4a3d6c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:04.232 7fda4a3d6c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.259 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:04.260 7fda4a3d6ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.260 7fda4a3d6ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:04.260 7fda4a3d6ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.265 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:04.265 7fda4a3d6ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.265 7fda4a3d6ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:04.266 7fda4a3d6ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.266 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:42:04.266 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:42:04.266 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:42:04.267 7fda4a3d6e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:04.267 7fda4a3440e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.267 7fda4a3440e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:04.268 7fda4a3440e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.268 7fda4a3440e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.268 7fda4a3440e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:04.268 7fda4a3440e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.270 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:42:04.275 7fda4a3d6e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:04.276 7fda4a344d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.276 7fda4a344d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:04.276 7fda4a344d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.298 7fda4a4c37c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:42:04.300 7fda4ba9a400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.300 7fda4ba9a400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:04.300 7fda4ba9a400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:04.327 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:04.328 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:04.328 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:04.328 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:04.329 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:04.329 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:04.590 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:04.590 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:04.591 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:04.591 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:04.591 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:04.592 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:05.139 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:05.244 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:05.244 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:05.244 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:05.245 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:05.245 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:05.246 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:06.076 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:06.077 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:06.077 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:06.078 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:06.078 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:06.078 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:06.139 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:07.140 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:07.622 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:07.622 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:07.623 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:07.623 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:07.623 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:07.624 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:08.135 7fda4a3444a0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:08.136 7fda4a3444a0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:08.137 7fda4a3444a0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:08.137 7fda4a3444a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:08.137 7fda4a3444a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:08.137 7fda4a3444a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:08.137 7fda4a3444a0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:08.138 7fda4a3444a0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:42:08.138 7fda4a3444a0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:08.138 7fda4a3444a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:08.138 7fda4a3444a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:08.139 7fda4a344d60 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:42:08.140 7fda4a344d60 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:42:08.141 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:09.143 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:09.653 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:09.654 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:09.654 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:09.655 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:09.655 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:09.656 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:10.144 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:11.145 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:12.146 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:13.147 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:14.148 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:14.698 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:14.699 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:14.699 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:14.699 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:14.699 7fda4a472a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:14.700 7fda4a472a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:15.150 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:16.151 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:17.152 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:18.153 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:19.155 7fda4a4c3f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:19.220 7fda4a472a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:42:19.220 7fda4a472a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:42:19.221 7fda4a472a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:19.395 7fda4a4c37c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:42:19.395 7fda4f5179c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:42:19.396 7fda4f5179c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:42:19.396 7fda4a472860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:42:19.396 7fda4a472860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:42:19.410 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:42:19.411 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:42:19.411 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:42:19.411 7fda4f5179c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:42:29.219 7fdecd226800 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:42:29.219 7fdecd226800 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:42:29.219 7fdecd226800 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:42:29.223 7fdecd226800 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:42:29.223 7fdec879c2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:42:29.223 7fdec879c2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:42:29.224 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:42:29.224 7fdec879c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:42:29.224 7fdec879c7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:42:29.224 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:42:29.225 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.225 7fdec879c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:42:29.225 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.225 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.225 7fdec879c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353749224748 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:42:29.225 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.226 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.226 7fdec879c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:29.226 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.226 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.226 7fdec879c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:29.227 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.227 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.227 7fdec879c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:29.227 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.227 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.227 7fdec879c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:29.227 7fdec879c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.227 7fdec879ce00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:29.228 7fdec879ce00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:42:29.229 7fdec879cc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:42:29.230 7fdec874c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.230 7fdec874c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:29.230 7fdec874c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.230 7fdec874c540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.230 7fdec874c540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:29.230 7fdec874c540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.313 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:42:29.324 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:42:29.325 7fdeca8b2040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.325 7fdeca8b2040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:29.325 7fdeca8b2040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.325 7fdeca8b2040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.325 7fdeca8b2040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:29.325 7fdeca8b2040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.327 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:29.328 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.328 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:29.329 7fdec874c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.330 7fdec874c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:29.330 7fdec874c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.331 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.331 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:29.332 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:42:29.333 7fdec874c7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.333 7fdec874c7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:29.333 7fdec874c7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.333 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.335 7fdeca8b2040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.335 7fdeca8b2040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:29.335 7fdeca8b2040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.347 7fdeca8b2c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.348 7fdeca8b2c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:29.348 7fdeca8b2c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.381 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:29.383 7fdeca8b2ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.383 7fdeca8b2ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:29.383 7fdeca8b2ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.387 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:29.388 7fdeca8b2ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.388 7fdeca8b2ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:29.388 7fdeca8b2ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.389 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:42:29.389 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:42:29.389 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:42:29.390 7fdeca8b2e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:29.391 7fdec9d6f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.391 7fdec9d6f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:29.391 7fdec9d6f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.391 7fdec9d6f0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.391 7fdec9d6f0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:29.391 7fdec9d6f0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.396 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:42:29.399 7fdeca8b2e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:29.399 7fdec9d6f9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.400 7fdec9d6f9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:29.400 7fdec9d6f9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.435 7fdec879c7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:42:29.438 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.438 7fdec9d4e400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:29.438 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:29.442 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:29.442 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.442 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:29.443 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.443 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:29.444 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.589 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:29.590 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.590 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:29.591 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.591 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:29.591 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.872 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:29.873 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.873 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:29.874 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:29.874 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:29.874 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:30.229 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:30.388 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:30.389 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:30.389 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:30.390 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:30.390 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:30.391 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:31.118 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:31.119 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:31.119 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:31.120 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:31.120 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:31.120 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:31.230 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:32.231 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:32.837 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:32.838 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:32.838 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:32.838 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:32.838 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:32.839 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:33.224 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:33.226 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:33.226 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:33.226 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:33.226 7fdec9d4e400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:33.226 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:33.226 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:33.226 7fdec9d4e400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:42:33.227 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:33.227 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:33.227 7fdec9d4e400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:33.227 7fdec9d4e900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:42:33.228 7fdec9d4e900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:42:33.232 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:34.233 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:35.234 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:36.005 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:36.006 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:36.006 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:36.007 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:36.007 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:36.007 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:36.236 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:37.237 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:38.238 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:39.239 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:40.240 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:41.242 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:42.242 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:43.199 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:43.200 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:43.200 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:43.201 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:43.201 7fdec874ca40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:43.201 7fdec874ca40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:43.244 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:44.244 7fdec879cf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:44.307 7fdec874ca40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:42:44.307 7fdec874ca40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:42:44.308 7fdec874ca40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:44.456 7fdec879c7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:42:44.456 7fdecd226800 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:42:44.456 7fdecd226800 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:42:44.456 7fdec874c860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:42:44.456 7fdec874c860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:42:44.470 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:42:44.471 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:42:44.471 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:42:44.471 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:42:44.471 7fdecd226800 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:42:55.425 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:42:55.425 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:42:55.426 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:42:55.430 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:42:55.430 7fc4c12d72c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:42:55.430 7fc4c12d72c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:42:55.431 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:42:55.431 7fc4c12d77c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:42:55.432 7fc4c12d77c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:42:55.432 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:42:55.432 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.432 7fc4c12d77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:42:55.432 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.432 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.432 7fc4c12d77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353775432016 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:42:55.432 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.433 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.434 7fc4c12d77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:55.435 7fc4c12d77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.435 7fc4c12d7e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:55.435 7fc4c12d7e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:42:55.436 7fc4c12d7c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:42:55.437 7fc4c1287540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.437 7fc4c1287540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:42:55.437 7fc4c1287540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.437 7fc4c1287540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.437 7fc4c1287540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:55.437 7fc4c1287540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.532 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:42:55.544 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:42:55.546 7fc4c1223040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.546 7fc4c1223040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:55.547 7fc4c1223040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.547 7fc4c1223040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.547 7fc4c1223040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:55.547 7fc4c1223040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.549 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:55.551 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:55.551 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:55.552 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:55.552 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:55.553 7fc4c12877c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.553 7fc4c12877c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:55.553 7fc4c12877c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.554 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:42:55.555 7fc4c12877c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.555 7fc4c12877c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:55.555 7fc4c12877c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.557 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:55.557 7fc4c12877c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.557 7fc4c12877c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:55.557 7fc4c12877c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.571 7fc4c1223040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.571 7fc4c1223040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:55.571 7fc4c1223040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.602 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:55.603 7fc4c1223c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.603 7fc4c1223c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:42:55.603 7fc4c1223c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.607 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:55.608 7fc4c1223c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.608 7fc4c1223c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:42:55.608 7fc4c1223c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.608 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:42:55.609 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:42:55.609 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:42:55.609 7fc4c1223e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:42:55.610 7fc4c115a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.610 7fc4c115a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:55.610 7fc4c115a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.610 7fc4c115a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.610 7fc4c115a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:55.610 7fc4c115a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.616 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:42:55.620 7fc4c1223e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:42:55.621 7fc4c115a860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.621 7fc4c115a860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:42:55.622 7fc4c115a860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.691 7fc4c12d77c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:42:55.693 7fc4c296a400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.694 7fc4c296a400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:55.694 7fc4c296a400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:55.708 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:55.708 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:55.708 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:55.709 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:55.709 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:55.709 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:55.998 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:55.998 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:55.999 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:55.999 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:56.000 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:56.000 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:56.436 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:56.484 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:56.485 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:56.485 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:56.485 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:56.485 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:56.486 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:57.295 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:57.296 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:57.296 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:57.297 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:57.297 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:57.297 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:57.438 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:58.439 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:42:58.908 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:42:58.909 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:58.909 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:42:58.910 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:58.910 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:42:58.910 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:42:59.432 7fc4c296a400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:59.434 7fc4c296a400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:59.434 7fc4c296a400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:59.434 7fc4c296a400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:59.434 7fc4c296a400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:42:59.434 7fc4c296a400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:59.434 7fc4c296a400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:59.434 7fc4c296a400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:42:59.435 7fc4c296a400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:59.435 7fc4c296a400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:59.435 7fc4c296a400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:59.438 7fc4c296a900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:42:59.439 7fc4c296a900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:42:59.440 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:00.441 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:01.442 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:02.403 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:02.404 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:02.404 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:02.404 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:02.404 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:02.405 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:02.443 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:03.444 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:04.446 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:05.447 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:06.448 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:07.449 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:07.454 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:07.455 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:07.455 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:07.455 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:07.455 7fc4c1287a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:07.456 7fc4c1287a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:08.450 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:09.452 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:10.454 7fc4c12d7f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:10.469 7fc4c1287a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:43:10.469 7fc4c1287a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:43:10.470 7fc4c1287a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:10.779 7fc4c12d77c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:43:10.779 7fc4c5d5d180 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:43:10.779 7fc4c5d5d180 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:43:10.780 7fc4c1287680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:43:10.780 7fc4c1287680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:43:10.808 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:43:10.809 7fc4c5d5d180 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:43:20.899 7f0eee78d100 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:43:20.899 7f0eee78d100 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:43:20.899 7f0eee78d100 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:43:20.905 7f0eee78d100 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:43:20.907 7f0ee9d0b2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:43:20.907 7f0ee9d0b2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:43:20.907 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:43:20.908 7f0ee9d0b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:43:20.908 7f0ee9d0b7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:43:20.908 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:43:20.908 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.909 7f0ee9d0b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:43:20.909 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.909 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.909 7f0ee9d0b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353800908659 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:43:20.909 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.910 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.912 7f0ee9d0b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:20.912 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.912 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.912 7f0ee9d0b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:20.912 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.912 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.914 7f0ee9d0b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:20.914 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.914 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.914 7f0ee9d0b7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:20.914 7f0ee9d0b7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.915 7f0ee9d0be00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:20.915 7f0ee9d0be00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:43:20.917 7f0ee9d0bc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:43:20.918 7f0ee9cbb540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.919 7f0ee9cbb540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:20.919 7f0ee9cbb540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.919 7f0ee9cbb540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.920 7f0ee9cbb540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:20.920 7f0ee9cbb540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.989 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:43:21.000 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:43:21.015 7f0ee9c16040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.016 7f0ee9c16040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:21.016 7f0ee9c16040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.016 7f0ee9c16040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.016 7f0ee9c16040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:21.016 7f0ee9c16040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.017 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:21.018 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.018 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:21.020 7f0ee9cbb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.021 7f0ee9cbb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:21.021 7f0ee9cbb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.022 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.022 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:21.024 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.029 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:43:21.030 7f0ee9cbb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.030 7f0ee9cbb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:21.030 7f0ee9cbb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.031 7f0ee9cbb7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.033 7f0ee9cbb7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:21.034 7f0ee9cbb7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.052 7f0ee9c16c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.052 7f0ee9c16c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:21.052 7f0ee9c16c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.084 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:21.085 7f0ee9c16ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.085 7f0ee9c16ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:21.085 7f0ee9c16ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.088 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:21.089 7f0ee9c16ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.089 7f0ee9c16ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:21.089 7f0ee9c16ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.090 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:43:21.090 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:43:21.090 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:43:21.091 7f0ee9c16e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:21.091 7f0ee9b8d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.091 7f0ee9b8d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:21.092 7f0ee9b8d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.092 7f0ee9b8d0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.092 7f0ee9b8d0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:21.092 7f0ee9b8d0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.096 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:43:21.099 7f0ee9c16e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:21.100 7f0ee9b8d860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.100 7f0ee9b8d860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:21.100 7f0ee9b8d860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.101 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:21.102 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.102 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:21.103 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.103 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:21.110 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.166 7f0ee9d0b7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:43:21.172 7f0eeb39c180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.173 7f0eeb39c180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:21.174 7f0eeb39c180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:21.302 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:21.303 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.303 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:21.304 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.304 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:21.304 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.664 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:21.665 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.665 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:21.666 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.666 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:21.666 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:21.917 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:22.479 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:22.480 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:22.480 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:22.480 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:22.480 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:22.481 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:22.918 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:23.920 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:24.303 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:24.303 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:24.304 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:24.304 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:24.304 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:24.305 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:24.908 7f0eeb39c680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:43:24.910 7f0eeb39c680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:43:24.910 7f0eeb39c680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:43:24.910 7f0eeb39c680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:24.910 7f0eeb39c680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:24.910 7f0eeb39c680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:24.910 7f0eeb39c680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:43:24.910 7f0eeb39c680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:43:24.911 7f0eeb39c680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:43:24.911 7f0eeb39c680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:24.911 7f0eeb39c680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:24.911 7f0eeb39c900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:43:24.912 7f0eeb39c900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:43:24.921 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:25.922 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:26.621 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:26.622 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:26.622 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:26.623 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:26.623 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:26.623 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:26.922 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:27.924 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:28.925 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:29.925 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:30.926 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:31.401 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:31.402 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:31.402 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:31.402 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:31.403 7f0ee9cbba40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:31.403 7f0ee9cbba40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:31.927 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:32.929 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:33.930 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:34.932 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:35.923 7f0ee9cbba40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:43:35.924 7f0ee9cbba40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:43:35.924 7f0ee9cbba40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:35.934 7f0ee9d0bf40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:36.216 7f0ee9d0b7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:43:36.216 7f0eee78d100 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:43:36.216 7f0eee78d100 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:43:36.216 7f0ee9cbb680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:43:36.216 7f0ee9cbb680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:43:36.230 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:43:36.230 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:43:36.230 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:43:36.231 7f0eee78d100 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:43:47.546 7f1251c56680 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:43:47.546 7f1251c56680 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:43:47.546 7f1251c56680 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:43:47.560 7f1251c56680 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:43:47.561 7f124d1cf2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:43:47.561 7f124d1cf2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:43:47.561 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:43:47.562 7f124d1cf7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:43:47.562 7f124d1cf7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:43:47.562 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:43:47.562 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.562 7f124d1cf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:43:47.562 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.562 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.562 7f124d1cf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353827562268 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:43:47.562 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.565 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.565 7f124d1cf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:47.565 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.565 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.565 7f124d1cf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:47.566 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.566 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.566 7f124d1cf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:43:47.566 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.566 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.566 7f124d1cf7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:47.566 7f124d1cf7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.581 7f124d1cfe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:47.581 7f124d1cfe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:43:47.582 7f124d1cfc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:43:47.583 7f124d17e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.583 7f124d17e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:43:47.583 7f124d17e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.583 7f124d17e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.583 7f124d17e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:47.583 7f124d17e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.750 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:43:47.773 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:43:47.778 7f124e045040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.780 7f124e045040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:47.780 7f124e045040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.780 7f124e045040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.781 7f124e045040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:47.781 7f124e045040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.788 7f124e045040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.789 7f124e045040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:47.790 7f124e045040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.792 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:47.795 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:47.795 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:47.797 7f124d17e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.798 7f124d17e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:47.798 7f124d17e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.801 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:47.801 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:47.803 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:43:47.803 7f124d17e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.804 7f124d17e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:47.804 7f124d17e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.804 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:47.833 7f124e045c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.835 7f124e045c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:47.835 7f124e045c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.856 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:47.857 7f124e045ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.857 7f124e045ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:43:47.857 7f124e045ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.860 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:47.860 7f124e045ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.860 7f124e045ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:43:47.860 7f124e045ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.861 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:43:47.861 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:43:47.861 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:43:47.861 7f124e045e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:43:47.862 7f124d0480e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.862 7f124d0480e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:47.862 7f124d0480e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.862 7f124d0480e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.862 7f124d0480e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:47.862 7f124d0480e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.867 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:43:47.868 7f124e045e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:43:47.869 7f124d048860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.869 7f124d048860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:43:47.869 7f124d048860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.933 7f124d1cf7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:43:47.940 7f124e865400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.941 7f124e865400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:47.941 7f124e865400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:47.960 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:47.961 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:47.961 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:47.961 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:47.961 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:47.962 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:48.245 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:48.246 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:48.246 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:48.247 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:48.247 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:48.247 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:48.582 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:48.739 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:48.740 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:48.740 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:48.741 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:48.741 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:48.742 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:49.583 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:49.934 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:49.935 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:49.935 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:49.935 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:49.936 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:49.936 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:50.584 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:51.562 7f124e865400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:43:51.563 7f124e865400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:43:51.563 7f124e865400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:43:51.563 7f124e865400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:51.564 7f124e865400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:43:51.564 7f124e865400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:51.564 7f124e865400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:43:51.564 7f124e865400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:43:51.564 7f124e865400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:43:51.564 7f124e865400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:51.564 7f124e865400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:51.565 7f124e865900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:43:51.566 7f124e865900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:43:51.585 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:51.669 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:51.670 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:51.670 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:51.671 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:51.671 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:51.671 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:52.587 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:53.588 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:54.534 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:54.535 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:54.535 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:54.536 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:54.536 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:54.536 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:54.589 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:55.590 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:56.591 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:57.593 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:58.594 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:43:58.632 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:43:58.633 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:58.633 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:43:58.634 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:58.634 7f124d17ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:43:58.634 7f124d17ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:43:59.595 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:00.597 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:01.598 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:02.599 7f124d1cff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:02.652 7f124d17ea40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:44:02.652 7f124d17ea40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:44:02.653 7f124d17ea40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:02.958 7f124d1cf7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:44:02.959 7f1251c56680 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:44:02.959 7f1251c56680 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:44:02.959 7f124d17e680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:44:02.959 7f124d17e680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:44:03.008 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:44:03.009 7f1251c56680 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:44:12.907 7fc525b48dc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:44:12.907 7fc525b48dc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:44:12.907 7fc525b48dc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:44:12.911 7fc525b48dc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:44:12.911 7fc5210c02c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:44:12.912 7fc5210c02c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:44:12.912 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:44:12.913 7fc5210c07c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:44:12.913 7fc5210c07c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:44:12.913 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:44:12.913 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.913 7fc5210c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:44:12.913 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.914 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.914 7fc5210c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353852913367 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:44:12.914 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.915 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.915 7fc5210c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:12.915 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.915 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.915 7fc5210c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:12.915 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.916 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.916 7fc5210c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:12.916 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.916 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.916 7fc5210c07c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:12.916 7fc5210c07c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.916 7fc5210c0f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:44:12.917 7fc5210c0f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:44:12.918 7fc5210c0e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:44:12.918 7fc5210707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.918 7fc5210707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:44:12.918 7fc5210707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.919 7fc5210707c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.919 7fc5210707c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:12.919 7fc5210707c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.982 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:44:12.991 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:44:12.992 7fc520fcb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.993 7fc520fcb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:12.993 7fc520fcb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.993 7fc520fcb0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.993 7fc520fcb0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:12.993 7fc520fcb0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.994 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:12.994 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:12.994 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:12.995 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:12.995 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:12.996 7fc5210709a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.996 7fc5210709a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:12.996 7fc5210709a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.998 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:12.998 7fc5210709a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:12.998 7fc5210709a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:12.998 7fc5210709a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.000 7fc5210709a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.000 7fc5210709a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:13.000 7fc5210709a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.011 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:44:13.012 7fc5210709a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.012 7fc5210709a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:13.012 7fc5210709a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.050 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:44:13.051 7fc520fcbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.052 7fc520fcbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:13.052 7fc520fcbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.056 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:44:13.056 7fc520fcbc20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.056 7fc520fcbc20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:13.056 7fc520fcbc20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.057 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:44:13.057 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:44:13.057 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:44:13.058 7fc520fcbf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:44:13.059 7fc52104f220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.059 7fc52104f220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:13.059 7fc52104f220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.059 7fc52104f220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.060 7fc52104f220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:13.060 7fc52104f220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.065 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:44:13.069 7fc520fcbf40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:44:13.069 7fc52104f9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.070 7fc52104f9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:13.070 7fc52104f9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.096 7fc5210c07c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:44:13.098 7fc522757400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.098 7fc522757400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:44:13.098 7fc522757400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:13.112 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:13.113 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:13.113 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:13.113 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:13.113 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:13.114 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:13.334 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:13.335 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:13.335 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:13.335 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:13.335 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:13.336 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:13.774 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:13.775 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:13.775 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:13.775 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:13.775 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:13.776 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:13.918 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:14.557 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:14.558 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:14.558 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:14.559 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:14.559 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:14.559 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:14.920 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:15.921 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:16.576 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:16.577 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:16.577 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:16.578 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:16.578 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:16.578 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:16.914 7fc52104f900 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:44:16.915 7fc52104f900 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:44:16.915 7fc52104f900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:44:16.915 7fc52104f900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:16.915 7fc52104f900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:44:16.915 7fc52104f900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:16.915 7fc52104f900 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:44:16.915 7fc52104f900 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:44:16.916 7fc52104f900 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:44:16.916 7fc52104f900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:16.916 7fc52104f900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:16.916 7fc52104f7c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:44:16.918 7fc52104f7c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:44:16.922 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:17.923 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:18.924 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:19.925 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:20.926 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:21.350 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:21.351 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:21.351 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:21.352 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:21.352 7fc521070c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:21.352 7fc521070c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:21.928 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:22.928 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:23.930 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:24.931 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:25.932 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:26.933 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:27.934 7fc5210702c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:27.982 7fc521070c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:44:27.982 7fc521070c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:44:27.983 7fc521070c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:28.155 7fc5210c07c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:44:28.155 7fc525b48dc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:44:28.155 7fc525b48dc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:44:28.155 7fc521070b80 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:44:28.156 7fc521070b80 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:44:28.176 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:44:28.176 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:44:28.176 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:44:28.176 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:44:28.176 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:44:28.176 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:44:28.176 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:44:28.177 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:44:28.177 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:44:28.177 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:44:28.177 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:44:28.177 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:44:28.177 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:44:28.177 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:44:28.177 7fc525b48dc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:44:37.647 7f17197470c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:44:37.647 7f17197470c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:44:37.647 7f17197470c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:44:37.650 7f17197470c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:44:37.651 7f1714cd62c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:44:37.651 7f1714cd62c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:44:37.651 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:44:37.651 7f1714cd67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:44:37.652 7f1714cd67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:44:37.652 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:44:37.652 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.652 7f1714cd67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:44:37.652 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.652 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.652 7f1714cd67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353877651997 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:44:37.652 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.653 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.653 7f1714cd67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:37.653 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.653 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.653 7f1714cd67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:37.653 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.654 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.654 7f1714cd67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:44:37.654 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.654 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.654 7f1714cd67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:37.654 7f1714cd67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.654 7f1714cd6e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:44:37.655 7f1714cd6e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:44:37.656 7f1714cd6c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:44:37.656 7f1714c86540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.656 7f1714c86540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:44:37.656 7f1714c86540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.656 7f1714c86540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.657 7f1714c86540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:37.657 7f1714c86540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.775 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:44:37.784 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:44:37.785 7f1714c68040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.786 7f1714c68040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:37.786 7f1714c68040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.786 7f1714c68040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.786 7f1714c68040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:37.786 7f1714c68040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.787 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:37.794 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:37.794 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:37.798 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:44:37.799 7f1714c867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.799 7f1714c867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:37.799 7f1714c867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.799 7f1714c867c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.799 7f1714c867c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:37.800 7f1714c867c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.800 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:37.800 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:37.819 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:37.821 7f1714c68c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.821 7f1714c68c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:37.821 7f1714c68c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.829 7f1714c68c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.829 7f1714c68c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:37.829 7f1714c68c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.848 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:44:37.849 7f1714c68ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.849 7f1714c68ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:44:37.849 7f1714c68ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.853 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:44:37.853 7f1714c68ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.853 7f1714c68ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:44:37.853 7f1714c68ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.854 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:44:37.854 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:44:37.854 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:44:37.854 7f1714c68e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:44:37.855 7f1714b590e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.855 7f1714b590e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:37.855 7f1714b590e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.855 7f1714b590e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.855 7f1714b590e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:37.855 7f1714b590e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.859 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:44:37.872 7f1714c68e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:44:37.873 7f1714b59860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.873 7f1714b59860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:44:37.873 7f1714b59860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.924 7f1714cd67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:44:37.926 7f17162aa180 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.926 7f17162aa180 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:44:37.927 7f17162aa180 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:37.958 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:37.959 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:37.959 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:37.959 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:37.959 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:37.960 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:38.201 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:38.202 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:38.202 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:38.203 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:38.203 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:38.203 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:38.656 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:38.786 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:38.787 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:38.787 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:38.788 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:38.788 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:38.789 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:39.657 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:39.759 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:39.760 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:39.760 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:39.761 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:39.761 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:39.761 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:40.659 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:41.036 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:41.037 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.037 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:41.038 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.038 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:41.039 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:41.652 7f17162aa680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:44:41.653 7f17162aa680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:44:41.653 7f17162aa680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:44:41.653 7f17162aa680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:41.653 7f17162aa680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:44:41.653 7f17162aa680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:41.654 7f17162aa680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:44:41.654 7f17162aa680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:44:41.654 7f17162aa680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:44:41.654 7f17162aa680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:41.654 7f17162aa680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:41.655 7f17162aa900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:44:41.656 7f17162aa900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:44:41.660 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:42.661 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:43.662 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:43.776 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:43.777 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:43.777 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:43.778 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:43.778 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:43.778 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:44.664 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:45.665 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:46.666 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:47.667 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:48.668 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:49.670 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:50.671 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:51.203 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:44:51.216 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:51.216 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:44:51.217 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:51.217 7f1714c86a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:44:51.217 7f1714c86a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:44:51.672 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:52.673 7f1714cd6f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:44:52.727 7f1714c86a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:44:52.727 7f1714c86a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:44:52.728 7f1714c86a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:52.948 7f1714cd67c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:44:52.948 7f17197470c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:44:52.948 7f17197470c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:44:52.949 7f1714c86680 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:44:52.949 7f1714c86680 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:44:52.966 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:44:52.966 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:44:52.966 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:44:52.966 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:44:52.966 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:44:52.966 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:44:52.967 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:44:52.967 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:44:52.967 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:44:52.967 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:44:52.967 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:44:52.967 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:44:52.967 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:44:52.967 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:44:52.967 7f17197470c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:45:02.043 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:45:02.043 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:45:02.043 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:45:02.047 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:45:02.047 7f2e2b12a2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:45:02.047 7f2e2b12a2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:45:02.047 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:45:02.048 7f2e2b12a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:45:02.048 7f2e2b12a7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:45:02.048 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:45:02.048 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.048 7f2e2b12a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:45:02.048 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.048 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.048 7f2e2b12a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353902048302 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:45:02.048 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.049 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.049 7f2e2b12a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:02.049 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.049 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.049 7f2e2b12a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:02.049 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.049 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.050 7f2e2b12a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:02.050 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.050 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.050 7f2e2b12a7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:02.050 7f2e2b12a7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.050 7f2e2b12ae00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:02.050 7f2e2b12ae00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:45:02.051 7f2e2b12ac20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:45:02.052 7f2e2b0da540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.052 7f2e2b0da540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:02.052 7f2e2b0da540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.052 7f2e2b0da540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.052 7f2e2b0da540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:02.053 7f2e2b0da540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.106 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:45:02.114 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:45:02.115 7f2e2b036040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.116 7f2e2b036040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:02.116 7f2e2b036040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.116 7f2e2b036040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.116 7f2e2b036040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:02.116 7f2e2b036040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.117 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:02.118 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.118 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:02.119 7f2e2b0da7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.119 7f2e2b0da7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:02.119 7f2e2b0da7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.119 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.119 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:02.120 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.121 7f2e2b0da7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.121 7f2e2b0da7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:02.121 7f2e2b0da7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.122 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:45:02.122 7f2e2b0da7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.122 7f2e2b0da7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:02.122 7f2e2b0da7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.131 7f2e2b036c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.132 7f2e2b036c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:02.132 7f2e2b036c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.156 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:02.156 7f2e2b036ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.157 7f2e2b036ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:02.157 7f2e2b036ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.162 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:02.162 7f2e2b036ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.162 7f2e2b036ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:02.163 7f2e2b036ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.163 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:45:02.163 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:45:02.163 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:45:02.164 7f2e2b036e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:02.164 7f2e2afaa0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.165 7f2e2afaa0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:02.165 7f2e2afaa0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.165 7f2e2afaa0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.165 7f2e2afaa0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:02.165 7f2e2afaa0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.169 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:45:02.173 7f2e2b036e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:02.174 7f2e2afaaae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.174 7f2e2afaaae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:02.174 7f2e2afaaae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.207 7f2e2b12a7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:45:02.211 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.211 7f2e2afaa5e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:02.211 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.212 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:02.213 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.213 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:02.214 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.214 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:02.215 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.356 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:02.357 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.357 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:02.357 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.357 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:02.358 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.706 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:02.706 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.706 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:02.707 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:02.707 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:02.707 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:03.051 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:03.438 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:03.439 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:03.439 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:03.440 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:03.440 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:03.440 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:04.053 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:04.448 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:04.449 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:04.449 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:04.449 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:04.450 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:04.450 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:05.053 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:06.049 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:06.050 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:06.050 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:06.050 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:06.050 7f2e2afaa5e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:06.050 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:06.050 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:06.051 7f2e2afaa5e0 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:45:06.051 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:06.051 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:06.051 7f2e2afaa5e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:06.051 7f2e2c6fc2c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:45:06.052 7f2e2c6fc2c0 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:45:06.055 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:06.309 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:06.310 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.310 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:06.311 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:06.311 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:06.311 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:07.055 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:08.057 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:09.058 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:09.926 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:09.927 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:09.927 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:09.928 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:09.928 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:09.928 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:10.059 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:11.060 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:12.061 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:13.062 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:14.063 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:15.064 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:16.066 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:16.768 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:16.769 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:16.769 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:16.770 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:16.770 7f2e2b0daa40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:16.770 7f2e2b0daa40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:17.067 7f2e2b12af40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:17.074 7f2e2b0daa40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:45:17.074 7f2e2b0daa40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:45:17.075 7f2e2b0daa40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:17.279 7f2e2b12a7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:45:17.279 7f2e2fbbec00 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:45:17.279 7f2e2fbbec00 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:45:17.279 7f2e2b0da860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:45:17.280 7f2e2b0da860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:45:17.293 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:45:17.293 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:45:17.293 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:45:17.293 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:45:17.293 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:45:17.294 7f2e2fbbec00 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:45:26.737 7fe39f15bec0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:45:26.737 7fe39f15bec0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:45:26.737 7fe39f15bec0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:45:26.741 7fe39f15bec0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:45:26.741 7fe39a6d62c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:45:26.741 7fe39a6d62c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:45:26.742 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:45:26.742 7fe39a6d67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:45:26.742 7fe39a6d67c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:45:26.743 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:45:26.743 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.743 7fe39a6d67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:45:26.743 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.743 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.743 7fe39a6d67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353926742877 collector_name = collector_ip = collector_list = [ 10.0.0.189:8086, 10.0.0.50:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:45:26.743 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.744 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.744 7fe39a6d67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:26.744 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.744 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.745 7fe39a6d67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:26.745 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.745 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.745 7fe39a6d67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:26.745 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.745 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.745 7fe39a6d67c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:26.745 7fe39a6d67c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.746 7fe39a6d6f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:26.746 7fe39a6d6f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:45:26.747 7fe39a6d6e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:45:26.747 7fe39a6857c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.748 7fe39a6857c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:26.748 7fe39a6857c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.748 7fe39a6857c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.748 7fe39a6857c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:26.748 7fe39a6857c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.807 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:45:26.816 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:45:26.817 7fe39a5e10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.818 7fe39a5e10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:26.818 7fe39a5e10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.818 7fe39a5e10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.818 7fe39a5e10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:26.818 7fe39a5e10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.819 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:26.820 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:26.820 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:26.821 7fe39a6859a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.821 7fe39a6859a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:26.821 7fe39a6859a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.821 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:26.821 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:26.822 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:26.823 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:45:26.823 7fe39a6859a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.823 7fe39a6859a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:26.823 7fe39a6859a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.825 7fe39a5e10e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.825 7fe39a5e10e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:26.825 7fe39a5e10e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.833 7fe39a5e1d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.833 7fe39a5e1d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:26.833 7fe39a5e1d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.862 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:26.863 7fe39a5e1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.863 7fe39a5e1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:26.863 7fe39a5e1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.868 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:26.868 7fe39a5e1c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.868 7fe39a5e1c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:26.868 7fe39a5e1c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.869 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:45:26.869 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:45:26.869 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:45:26.870 7fe39a5e1f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:26.870 7fe39a55a220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.871 7fe39a55a220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:26.871 7fe39a55a220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.871 7fe39a55a220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.871 7fe39a55a220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:26.871 7fe39a55a220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.873 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:45:26.879 7fe39a5e1f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:26.879 7fe39a55aea0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.879 7fe39a55aea0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:26.879 7fe39a55aea0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.911 7fe39a6d67c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:45:26.914 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.914 7fe39bc9a400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:26.914 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:26.959 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:26.960 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:26.960 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:26.961 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:26.961 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:26.961 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:27.164 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:27.165 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:27.165 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:27.166 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:27.166 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:27.166 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:27.641 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:27.641 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:27.642 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:27.642 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:27.642 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:27.643 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:27.747 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:28.749 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:28.900 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:28.900 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:28.901 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:28.901 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:28.901 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:28.902 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:29.750 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:30.743 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:30.744 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:30.744 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:30.744 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:30.744 7fe39bc9a400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:30.744 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:30.744 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:30.744 7fe39bc9a400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:45:30.745 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:30.745 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:30.745 7fe39bc9a400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:30.745 7fe39bc9a900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:45:30.746 7fe39bc9a900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:45:30.751 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:31.358 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:31.359 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:31.359 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:31.359 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:31.359 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:31.360 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:31.752 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:32.754 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:33.755 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:34.757 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:35.758 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:36.463 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:36.464 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:36.464 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:36.465 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:36.465 7fe39a685c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:36.466 7fe39a685c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:36.759 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:37.762 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:38.764 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:39.765 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:40.766 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:41.768 7fe39a6852c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:41.790 7fe39a685c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:45:41.791 7fe39a685c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:45:41.792 7fe39a685c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:41.932 7fe39a6d67c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:45:41.932 7fe39f15bec0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:45:41.932 7fe39f15bec0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:45:41.932 7fe39a685860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:45:41.932 7fe39a685860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:45:41.949 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:45:41.950 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:45:41.950 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:45:41.950 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:45:41.950 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:45:41.950 7fe39f15bec0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:45:50.977 7f1b1f63df80 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:45:50.977 7f1b1f63df80 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:45:50.977 7f1b1f63df80 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:45:50.982 7f1b1f63df80 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:45:50.983 7f1b1abb52c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:45:50.983 7f1b1abb52c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:45:50.984 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:45:50.984 7f1b1abb57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:45:50.985 7f1b1abb57c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:45:50.985 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:45:50.985 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.985 7f1b1abb57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:45:50.985 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.985 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.986 7f1b1abb57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353950985156 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:45:50.986 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.987 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.987 7f1b1abb57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:50.987 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.987 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.987 7f1b1abb57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:50.987 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.987 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.987 7f1b1abb57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:45:50.987 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.988 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.988 7f1b1abb57c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:50.988 7f1b1abb57c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.988 7f1b1abb5e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:50.988 7f1b1abb5e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:45:50.989 7f1b1abb5c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:45:50.990 7f1b1ab64540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.990 7f1b1ab64540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:45:50.990 7f1b1ab64540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.990 7f1b1ab64540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:50.990 7f1b1ab64540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:50.990 7f1b1ab64540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.059 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:45:51.070 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:45:51.075 7f1b1ab05040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.076 7f1b1ab05040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:51.076 7f1b1ab05040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.076 7f1b1ab05040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.077 7f1b1ab05040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:51.077 7f1b1ab05040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.078 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:51.080 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:51.080 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:51.083 7f1b1ab647c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.084 7f1b1ab647c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:51.084 7f1b1ab647c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.085 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:51.085 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:51.086 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:45:51.087 7f1b1ab647c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.087 7f1b1ab647c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:51.087 7f1b1ab647c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.089 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:51.108 7f1b1ab05c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.109 7f1b1ab05c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:51.109 7f1b1ab05c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.117 7f1b1ab05c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.117 7f1b1ab05c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:51.117 7f1b1ab05c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.138 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:51.139 7f1b1ab05ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.139 7f1b1ab05ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:45:51.139 7f1b1ab05ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.143 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:51.143 7f1b1ab05ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.143 7f1b1ab05ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:45:51.144 7f1b1ab05ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.144 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:45:51.144 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:45:51.144 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:45:51.144 7f1b1ab05e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:45:51.145 7f1b1aa390e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.145 7f1b1aa390e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:51.145 7f1b1aa390e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.145 7f1b1aa390e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.145 7f1b1aa390e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:51.146 7f1b1aa390e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.149 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:45:51.152 7f1b1ab05e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:45:51.152 7f1b1aa39ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.153 7f1b1aa39ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:45:51.153 7f1b1aa39ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.179 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:51.180 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:51.180 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:51.180 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:51.181 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:51.182 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:51.183 7f1b1abb57c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:45:51.185 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.185 7f1b1aa39680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:51.185 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:51.421 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:51.422 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:51.422 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:51.422 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:51.422 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:51.423 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:51.989 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:52.043 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:52.044 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:52.044 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:52.044 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:52.044 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:52.045 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:52.946 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:52.947 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:52.947 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:52.947 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:52.947 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:52.948 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:52.990 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:53.991 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:54.810 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:54.810 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:54.811 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:54.811 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:54.812 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:54.812 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:54.985 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:54.986 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:54.986 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:54.986 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:54.987 7f1b1aa39680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:45:54.987 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:54.987 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:54.987 7f1b1aa39680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:45:54.987 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:54.987 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:54.987 7f1b1aa39680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:54.987 7f1b1ccb5180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:45:54.988 7f1b1ccb5180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:45:54.992 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:55.993 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:56.994 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:57.995 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:58.628 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:45:58.629 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:58.629 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:45:58.630 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:58.630 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:45:58.631 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:45:58.996 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:45:59.998 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:00.999 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:02.001 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:03.002 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:03.272 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:03.273 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:03.274 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:03.274 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:03.274 7f1b1ab64a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:03.275 7f1b1ab64a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:04.004 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:05.005 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:06.007 7f1b1abb5f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:06.088 7f1b1ab64a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:46:06.088 7f1b1ab64a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:46:06.089 7f1b1ab64a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:06.236 7f1b1abb57c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:46:06.237 7f1b1f63df80 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:46:06.237 7f1b1f63df80 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:46:06.237 7f1b1ab64860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:46:06.237 7f1b1ab64860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:46:06.251 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:46:06.252 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:46:06.252 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:46:06.252 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:46:06.252 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:46:06.252 7f1b1f63df80 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:46:16.112 7f1110f54c40 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:46:16.112 7f1110f54c40 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:46:16.112 7f1110f54c40 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:46:16.116 7f1110f54c40 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:46:16.117 7f110c4d12c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:46:16.117 7f110c4d12c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:46:16.117 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:46:16.117 7f110c4d17c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:46:16.118 7f110c4d17c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:46:16.118 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:46:16.118 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.118 7f110c4d17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:46:16.119 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.119 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.119 7f110c4d17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789353976118376 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:46:16.119 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.120 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.120 7f110c4d17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:16.120 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.120 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.120 7f110c4d17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:16.120 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.120 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.120 7f110c4d17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:16.121 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.121 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.121 7f110c4d17c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:16.121 7f110c4d17c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.121 7f110c4d1e00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:46:16.121 7f110c4d1e00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:46:16.122 7f110c4d1c20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:46:16.123 7f110c481540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.123 7f110c481540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:46:16.123 7f110c481540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.123 7f110c481540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.123 7f110c481540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:16.123 7f110c481540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.189 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:46:16.198 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:46:16.199 7f110e5e2040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.199 7f110e5e2040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:16.199 7f110e5e2040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.200 7f110e5e2040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.200 7f110e5e2040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:16.200 7f110e5e2040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.201 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:16.201 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:16.201 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:16.202 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:16.202 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:16.203 7f110c4817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.203 7f110c4817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:16.204 7f110c4817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.204 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:16.205 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:46:16.205 7f110c4817c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.205 7f110c4817c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:16.205 7f110c4817c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.207 7f110e5e2040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.207 7f110e5e2040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:16.207 7f110e5e2040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.216 7f110e5e2c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.216 7f110e5e2c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:16.216 7f110e5e2c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.245 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:46:16.246 7f110e5e2ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.246 7f110e5e2ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:16.246 7f110e5e2ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.250 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:46:16.250 7f110e5e2ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.250 7f110e5e2ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:16.250 7f110e5e2ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.250 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:46:16.251 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:46:16.251 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:46:16.251 7f110e5e2e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:46:16.252 7f110c4580e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.252 7f110c4580e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:16.252 7f110c4580e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.252 7f110c4580e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.252 7f110c4580e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:16.252 7f110c4580e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.257 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:46:16.259 7f110e5e2e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:46:16.259 7f110c4589a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.260 7f110c4589a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:16.260 7f110c4589a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.291 7f110c4d17c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:46:16.293 7f110da83400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.294 7f110da83400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:46:16.294 7f110da83400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:16.343 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:16.344 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:16.344 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:16.345 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:16.345 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:16.345 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:16.717 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:16.718 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:16.718 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:16.719 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:16.719 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:16.720 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:17.122 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:17.242 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:17.242 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:17.242 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:17.243 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:17.243 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:17.243 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:18.123 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:18.286 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:18.287 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:18.287 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:18.288 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:18.289 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:18.289 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:19.124 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:20.118 7f110da83400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:46:20.119 7f110da83400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:46:20.119 7f110da83400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:46:20.119 7f110da83400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:20.120 7f110da83400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:46:20.120 7f110da83400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:20.120 7f110da83400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:46:20.120 7f110da83400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:46:20.120 7f110da83400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:46:20.120 7f110da83400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:20.120 7f110da83400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:20.121 7f110da83900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:46:20.121 7f110da83900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:46:20.125 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:20.747 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:20.748 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:20.749 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:20.749 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:20.749 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:20.750 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:21.126 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:22.127 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:23.128 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:24.130 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:25.131 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:26.132 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:26.694 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:26.695 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:26.695 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:26.696 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:26.696 7f110c481a40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:26.696 7f110c481a40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:27.133 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:28.135 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:29.135 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:30.137 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:31.138 7f110c4d1f40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:31.216 7f110c481a40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:46:31.217 7f110c481a40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:46:31.217 7f110c481a40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:31.342 7f110c4d17c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:46:31.342 7f1110f54c40 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:46:31.343 7f1110f54c40 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:46:31.343 7f110c481860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:46:31.343 7f110c481860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:46:31.365 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:46:31.365 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:46:31.365 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:46:31.365 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:46:31.365 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:46:31.366 7f1110f54c40 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:46:41.014 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:46:41.014 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:46:41.014 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:46:41.018 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:46:41.018 7f07742172c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:46:41.018 7f07742172c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:46:41.019 7f07742177c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:46:41.019 7f07742177c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:46:41.019 7f07742177c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:46:41.019 7f07742177c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:46:41.020 7f07742177c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.020 7f07742177c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:46:41.020 7f07742177c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.020 7f07742177c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.020 7f07742177c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354001019782 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:46:41.020 7f07742177c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.021 7f0774217900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.021 7f0774217900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:41.021 7f0774217900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.021 7f0774217900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.022 7f0774217900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:41.022 7f0774217900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.022 7f0774217900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.022 7f0774217900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:46:41.022 7f0774217900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.022 7f0774217900 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.022 7f0774217900 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:41.022 7f0774217900 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.023 7f0774217f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:46:41.023 7f0774217f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:46:41.024 7f0774217e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:46:41.025 7f07741c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.025 7f07741c77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:46:41.025 7f07741c77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.025 7f07741c77c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.025 7f07741c77c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:41.025 7f07741c77c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.089 7f0774217900 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:46:41.097 7f0774217900 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:46:41.098 7f07740980e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.099 7f07740980e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:41.099 7f07740980e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.099 7f07740980e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.099 7f07740980e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:41.099 7f07740980e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.100 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:41.101 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:41.101 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:41.102 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:41.102 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:41.103 7f07741c79a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.103 7f07741c79a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:41.103 7f07741c79a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.104 7f0774217900 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:46:41.105 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:41.105 7f07741c79a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.105 7f07741c79a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:41.105 7f07741c79a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.106 7f07740980e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.107 7f07740980e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:41.107 7f07740980e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.116 7f0774098d60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.116 7f0774098d60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:41.116 7f0774098d60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.145 7f0774217900 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:46:41.145 7f0774098c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.146 7f0774098c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:46:41.146 7f0774098c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.149 7f0774217900 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:46:41.150 7f0774098c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.150 7f0774098c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:46:41.150 7f0774098c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.150 7f0774217900 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:46:41.150 7f0774217900 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:46:41.151 7f0774217900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:46:41.151 7f0774098f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:46:41.151 7f0776328220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.152 7f0776328220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:41.152 7f0776328220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.152 7f0776328220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.152 7f0776328220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:41.152 7f0776328220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.156 7f0774217900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:46:41.158 7f0774098f40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:46:41.159 7f07763289a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.159 7f07763289a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:46:41.159 7f07763289a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.182 7f0774217900 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:46:41.185 7f07757c8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.185 7f07757c8400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:46:41.185 7f07757c8400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:41.231 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:41.232 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:41.232 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:41.233 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:41.233 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:41.233 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:41.487 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:41.488 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:41.488 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:41.489 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:41.489 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:41.490 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:42.024 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:42.067 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:42.068 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:42.068 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:42.069 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:42.069 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:42.069 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:42.798 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:42.799 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:42.799 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:42.799 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:42.800 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:42.800 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:43.025 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:43.792 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:43.793 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:43.793 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:43.794 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:43.794 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:43.794 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:44.026 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:45.019 7f07757c8400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:46:45.021 7f07757c8400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:46:45.021 7f07757c8400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:46:45.021 7f07757c8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:45.021 7f07757c8400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:46:45.022 7f07757c8400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:45.022 7f07757c8400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:46:45.022 7f07757c8400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:46:45.022 7f07757c8400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:46:45.022 7f07757c8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:45.022 7f07757c8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:45.022 7f07757c8900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:46:45.023 7f07757c8900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:46:45.027 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:45.953 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:45.954 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:45.954 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:45.955 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:45.955 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:45.955 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:46.028 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:47.029 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:48.031 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:49.032 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:50.033 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:51.034 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:51.518 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:46:51.519 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:51.519 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:46:51.521 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:51.521 7f07741c7c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:46:51.521 7f07741c7c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:46:52.035 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:53.037 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:54.038 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:55.039 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:56.040 7f07741c7c20 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:46:56.040 7f07741c7c20 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:46:56.040 7f07741c72c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:46:56.041 7f07741c7c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:56.194 7f0774217900 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:46:56.195 7f0778ee8fc0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:46:56.195 7f0778ee8fc0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:46:56.195 7f07741c7860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:46:56.195 7f07741c7860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:46:56.208 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:46:56.209 7f0778ee8fc0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:47:05.926 7f332cc105c0 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:47:05.926 7f332cc105c0 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:47:05.926 7f332cc105c0 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:47:05.929 7f332cc105c0 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:47:05.930 7f332817f2c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:47:05.930 7f332817f2c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:47:05.931 7f332817f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:47:05.933 7f332817f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:47:05.933 7f332817f7c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:47:05.934 7f332817f7c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:47:05.934 7f332817f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.934 7f332817f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:47:05.934 7f332817f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.934 7f332817f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.934 7f332817f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354025933645 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.38:8086, 10.0.0.189:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:47:05.934 7f332817f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.936 7f332817f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.936 7f332817f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:05.936 7f332817f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.936 7f332817f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.936 7f332817f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:05.936 7f332817f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.936 7f332817f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.937 7f332817f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:05.937 7f332817f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.937 7f332817f7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.937 7f332817f7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:05.937 7f332817f7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.937 7f332817fe00 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:47:05.938 7f332817fe00 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:47:05.939 7f332817fc20 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:47:05.940 7f332812e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.940 7f332812e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:47:05.940 7f332812e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.940 7f332812e540 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:05.940 7f332812e540 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:05.940 7f332812e540 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.006 7f332817f7c0 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:47:06.016 7f332817f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:47:06.017 7f3328115040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.017 7f3328115040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:06.018 7f3328115040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.018 7f3328115040 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.018 7f3328115040 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:06.018 7f3328115040 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.019 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:06.020 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.020 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:06.022 7f332812e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.022 7f332812e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:06.022 7f332812e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.023 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.023 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:06.024 7f332817f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:47:06.024 7f332812e7c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.024 7f332812e7c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:06.024 7f332812e7c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.025 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.033 7f3328115c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.033 7f3328115c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:06.033 7f3328115c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.035 7f3328115c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.036 7f3328115c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:06.036 7f3328115c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.068 7f332817f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:47:06.069 7f3328115ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.070 7f3328115ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:06.070 7f3328115ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.074 7f332817f7c0 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:47:06.075 7f3328115ae0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.075 7f3328115ae0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:06.075 7f3328115ae0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.075 7f332817f7c0 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:47:06.076 7f332817f7c0 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:47:06.076 7f332817f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:47:06.077 7f3328115e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:47:06.077 7f33280020e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.078 7f33280020e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:06.078 7f33280020e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.078 7f33280020e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.078 7f33280020e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:06.078 7f33280020e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.083 7f332817f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:47:06.087 7f3328115e00 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:47:06.088 7f3328002860 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.088 7f3328002860 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:06.088 7f3328002860 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.117 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:06.119 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.120 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:06.121 7f332817f7c0 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:47:06.122 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.122 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:06.124 7f3328002680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.124 7f3328002680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:47:06.124 7f3328002680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:06.125 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.363 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:06.364 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.364 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:06.365 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.365 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:06.365 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.663 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:06.664 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.664 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:06.665 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.665 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:06.665 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:06.939 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:07.185 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:07.186 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:07.186 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:07.187 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:07.187 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:07.188 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:07.897 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:07.898 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:07.898 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:07.899 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:07.899 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:07.899 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:07.942 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:08.944 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:09.012 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:09.013 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:09.013 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:09.014 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:09.014 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:09.015 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:09.932 7f3328002680 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:47:09.933 7f3328002680 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:47:09.934 7f3328002680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:47:09.934 7f3328002680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:09.934 7f3328002680 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:47:09.934 7f3328002680 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:09.934 7f3328002680 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:47:09.934 7f3328002680 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:47:09.935 7f3328002680 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:47:09.935 7f3328002680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:09.935 7f3328002680 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:09.935 7f332a27f180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:47:09.936 7f332a27f180 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:47:09.945 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:10.946 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:10.989 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:10.990 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:10.990 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:10.990 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:10.990 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:10.991 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:11.948 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:12.949 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:13.951 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:14.567 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:14.567 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:14.568 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:14.568 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:14.568 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:14.569 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:14.952 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:15.953 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:16.954 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:17.955 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:18.956 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:20.000 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:20.811 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:20.812 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:20.812 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:20.812 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:20.812 7f332812ea40 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:20.813 7f332812ea40 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:21.000 7f332817ff40 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:21.015 7f332812ea40 [kazoo.client] [WARNING]: Failed connecting to Zookeeper within the connection retry policy. 09/14/2026 02:47:21.016 7f332812ea40 [kazoo.client] [INFO]: Zookeeper session closed, state: CLOSED 09/14/2026 02:47:21.016 7f332812ea40 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:21.212 7f332817f7c0 [contrail-alarm-gen] [ERROR]: Exception AttributeError connecting to Config DB. Arguments: ("module 'gevent._gevent_cevent' has no attribute 'Timeout'",): Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 497, in connect self._zk_client.start() File "/usr/local/lib/python3.9/site-packages/kazoo/client.py", line 673, in start raise self.handler.timeout_exception("Connection time-out") kazoo.handlers.gevent.SequentialGeventHandler.timeout_exception During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/opserver/config_handler.py", line 59, in start self._vnc_db = VncObjectDBClient(self._cassandra_cfg['servers'], File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_object_db.py", line 20, in __init__ self._object_db = vnc_cassandra.VncCassandraClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/vnc_cassandra.py", line 107, in __init__ self._cassandra_driver = driverClass(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 187, in __init__ super(CassandraDriverCQL, self).__init__(server_list, **options) File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 167, in __init__ self._init_cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/api.py", line 191, in _init_cluster self._Init_Cluster() File "/usr/local/lib/python3.9/site-packages/cfgm_common/datastore/drivers/cassandra_cql.py", line 403, in _Init_Cluster self._zk_client = ZookeeperClient( File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 478, in __init__ self.connect() File "/usr/local/lib/python3.9/site-packages/cfgm_common/zkclient.py", line 499, in connect except gevent.event.Timeout as e: AttributeError: module 'gevent._gevent_cevent' has no attribute 'Timeout' 09/14/2026 02:47:21.212 7f332cc105c0 [contrail-alarm-gen] [ERROR]: AlarmGen stopping everything 09/14/2026 02:47:21.212 7f332cc105c0 [contrail-alarm-gen] [ERROR]: Stopped http server 09/14/2026 02:47:21.212 7f332812e860 [contrail-alarm-gen] [INFO]: Processing event[EvStop] in state[ClientInit] 09/14/2026 02:47:21.212 7f332812e860 [contrail-alarm-gen] [INFO]: Unconsumed event[EvStop] in state[ClientInit] 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 1 of 5 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 1 of 5 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 1 of 5 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 2 of 5 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 2 of 5 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 2 of 5 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 3 of 5 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 3 of 5 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 3 of 5 09/14/2026 02:47:21.225 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 4 of 5 09/14/2026 02:47:21.226 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 4 of 5 09/14/2026 02:47:21.226 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 4 of 5 09/14/2026 02:47:21.226 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen killing 5 of 5 09/14/2026 02:47:21.226 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen joining 5 of 5 09/14/2026 02:47:21.226 7f332cc105c0 [contrail-alarm-gen] [INFO]: AlarmGen stopped 5 of 5 09/14/2026 02:47:30.674 7fdf6b789540 [contrail-alarm-gen] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:47:30.674 7fdf6b789540 [contrail-alarm-gen] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:47:30.674 7fdf6b789540 [contrail-alarm-gen] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:47:30.678 7fdf6b789540 [contrail-alarm-gen] [INFO]: Found extensions for set() 09/14/2026 02:47:30.678 7fdf667352c0 [contrail-alarm-gen] [ERROR]: Cannot write http_port 5995 to /tmp/contrail-alarm-gen.0.http_port 09/14/2026 02:47:30.678 7fdf667352c0 [contrail-alarm-gen] [ERROR]: Starting Introspect on HTTP Port 5995 09/14/2026 02:47:30.679 7fdf667357c0 [contrail-alarm-gen] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:47:30.679 7fdf667357c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 09/14/2026 02:47:30.679 7fdf667357c0 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:47:30.679 7fdf667357c0 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:47:30.679 7fdf667357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.679 7fdf667357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:47:30.679 7fdf667357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.680 7fdf667357c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.680 7fdf667357c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 5995 start_time = 1789354050679577 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:47:30.680 7fdf667357c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized] connection down >>, ] >> 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.682 7fdf66735c20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.683 7fdf66735c20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:30.683 7fdf66735c20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.683 7fdf66735f40 [contrail-alarm-gen] [DEBUG]: send alarmgen status : AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:47:30.683 7fdf66735f40 [contrail-alarm-gen] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:AlarmgenStatusTrace] map 09/14/2026 02:47:30.684 7fdf66735e00 [contrail-alarm-gen] [INFO]: Analytics Discovery zk start 09/14/2026 02:47:30.685 7fdf666e47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.685 7fdf666e47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] AlarmgenStatusTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. counters = [ << instance = 0 partitions = 0 keys = 0 updates = 0 table_stats = [ ] >>, ] alarmgens = [ cn-jenkins-deploy-platform-ansible-os-6277-2.:AnalyticsAlarm:contrail-alarm-gen:0, ] >> 09/14/2026 02:47:30.685 7fdf666e47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.685 7fdf666e47c0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.685 7fdf666e47c0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:30.685 7fdf666e47c0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.749 7fdf66735c20 [contrail-alarm-gen] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.189:5673//', 'pyamqp://guest:guest@10.0.0.50:5673//'] 09/14/2026 02:47:30.758 7fdf66735c20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection down 09/14/2026 02:47:30.760 7fdf6667a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.760 7fdf6667a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:30.760 7fdf6667a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.760 7fdf6667a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.761 7fdf6667a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.38:6379[Redis Instance initialized], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:30.761 7fdf6667a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.762 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:30.762 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:30.762 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:30.764 7fdf666e49a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.764 7fdf666e49a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:30.764 7fdf666e49a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.765 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:30.765 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:30.765 7fdf66735c20 [contrail-alarm-gen] [WARNING]: RabbitMQ drainer connection ESTABLISHED 09/14/2026 02:47:30.766 7fdf666e49a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.766 7fdf666e49a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.189:6379[Redis Instance initialized], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:30.766 7fdf666e49a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.766 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:30.768 7fdf6667a0e0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.768 7fdf6667a0e0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Initializing description = Redis Instance initialized >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Redis-UVE:10.0.0.50:6379[Redis Instance initialized], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:30.768 7fdf6667a0e0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.779 7fdf6667ad60 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.779 7fdf6667ad60 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:30.779 7fdf6667ad60 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.800 7fdf66735c20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:47:30.801 7fdf6667ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.801 7fdf6667ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[] connection down >>, ] >> 09/14/2026 02:47:30.801 7fdf6667ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.805 7fdf66735c20 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:47:30.805 7fdf6667ac20 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.805 7fdf6667ac20 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized] connection down >>, ] >> 09/14/2026 02:47:30.805 7fdf6667ac20 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.806 7fdf66735c20 [contrail-alarm-gen] [WARNING]: VNCCassandra started with driver cql 09/14/2026 02:47:30.806 7fdf66735c20 [contrail-alarm-gen] [WARNING]: initializing cassandra cluster 09/14/2026 02:47:30.806 7fdf66735c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster before create 09/14/2026 02:47:30.807 7fdf6667af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection down 09/14/2026 02:47:30.807 7fdf665ba220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.808 7fdf665ba220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:30.808 7fdf665ba220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.808 7fdf665ba220 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.808 7fdf665ba220 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:RabbitMQ[], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:30.808 7fdf665ba220 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.811 7fdf66735c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster created 09/14/2026 02:47:30.816 7fdf6667af40 [contrail-alarm-gen] [WARNING]: RabbitMQ publish connection ESTABLISHED 09/14/2026 02:47:30.816 7fdf665ba9a0 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.817 7fdf665ba9a0 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[] connection down >>, ] >> 09/14/2026 02:47:30.817 7fdf665ba9a0 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.858 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:30.861 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:30.862 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:30.863 7fdf66735c20 [contrail-alarm-gen] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 09/14/2026 02:47:30.865 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:30.865 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:30.867 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.867 7fdf67dc8400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:47:30.867 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:30.868 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:30.958 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:30.958 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:30.959 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:30.959 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:30.959 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:30.960 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:31.113 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:31.114 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:31.114 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:31.115 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:31.115 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:31.116 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:31.328 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:31.329 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:31.329 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:31.329 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:31.329 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:31.330 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:31.684 7fdf666e42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:31.767 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:31.768 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:31.768 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:31.769 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:31.769 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:31.769 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:32.559 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:32.560 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:32.560 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:32.560 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:32.560 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:32.561 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:32.685 7fdf666e42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:33.686 7fdf666e42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:33.851 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:33.852 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:33.852 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:33.853 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:33.853 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:33.854 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:34.679 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:47:34.681 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Session Event: TCP Connected 09/14/2026 02:47:34.681 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:47:34.682 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:34.682 7fdf67dc8400 [contrail-alarm-gen] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-2. process_status = [ << module_id = contrail-alarm-gen instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Redis-UVE name = 10.0.0.38:6379 server_addrs = [ 10.0.0.38:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.189:6379 server_addrs = [ 10.0.0.189:6379, ] status = Up >>, << type = Redis-UVE name = 10.0.0.50:6379 server_addrs = [ 10.0.0.50:6379, ] status = Up >>, << type = Zookeeper name = AlarmGenerator server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = Connection to Zookeeper initialized >>, << type = KafkaPub name = KafkaTopic status = Up description = All Kafka partitions are reachable >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.38:5673, 10.0.0.189:5673, 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.38:9041, 10.0.0.189:9041, 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.38:2181, 10.0.0.189:2181, 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Zookeeper:AlarmGenerator[Connection to Zookeeper initialized], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 09/14/2026 02:47:34.682 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:34.682 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:47:34.682 7fdf67dc8400 [contrail-alarm-gen] [DEBUG]: Send sandesh control message. uve type count # 7 09/14/2026 02:47:34.683 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:47:34.683 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:34.683 7fdf67dc8400 [contrail-alarm-gen] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:34.683 7fdf67dc8900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/14/2026 02:47:34.685 7fdf67dc8900 [contrail-alarm-gen] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/14/2026 02:47:34.687 7fdf666e42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:35.690 7fdf666e42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:36.694 7fdf666e42c0 [contrail-alarm-gen] [INFO]: UVE Done 09/14/2026 02:47:37.049 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.38(10.0.0.38):2181, use_ssl: False 09/14/2026 02:47:37.051 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.051 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.189(10.0.0.189):2181, use_ssl: False 09/14/2026 02:47:37.052 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused 09/14/2026 02:47:37.052 7fdf666e4c20 [kazoo.client] [INFO]: Connecting to 10.0.0.50(10.0.0.50):2181, use_ssl: False 09/14/2026 02:47:37.052 7fdf666e4c20 [kazoo.client] [WARNING]: Connection dropped: socket connection error: Connection refused